Attachment 660408 Details for Bug 885489 – /var/log/vdsm/vdsm.log (from hypervisor 2 of 2)

/var/log/vdsm/vdsm.log (from hypervisor 2 of 2)

vdsm.log-hyp2 (text/plain), 6.42 MB, created by Ian Pilcher on 2012-12-09 20:06:56 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Ian Pilcher

Created: 2012-12-09 20:06:56 UTC

Size: 6.42 MB

patch

obsolete

>Dummy-85::DEBUG::2012-12-08 14:01:02,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:02,324::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164923 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:04,331::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:04,370::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016029 s, 63.9 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:05,967::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:05,981::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000740607 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:06,377::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:06,432::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0286135 s, 35.8 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:08,439::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:08,493::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277097 s, 37.0 MB/s\n'; <rc> = 0
>Thread-127743::DEBUG::2012-12-08 14:01:09,071::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127743::DEBUG::2012-12-08 14:01:09,072::task::588::TaskManager.Task::(_updateState) Task=`7cabc832-0aad-4604-94d6-27e871a078ca`::moving from state init -> state preparing
>Thread-127743::INFO::2012-12-08 14:01:09,073::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127743::INFO::2012-12-08 14:01:09,073::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127743::DEBUG::2012-12-08 14:01:09,074::task::1172::TaskManager.Task::(prepare) Task=`7cabc832-0aad-4604-94d6-27e871a078ca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127743::DEBUG::2012-12-08 14:01:09,074::task::588::TaskManager.Task::(_updateState) Task=`7cabc832-0aad-4604-94d6-27e871a078ca`::moving from state preparing -> state finished
>Thread-127743::DEBUG::2012-12-08 14:01:09,075::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127743::DEBUG::2012-12-08 14:01:09,075::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127743::DEBUG::2012-12-08 14:01:09,076::task::978::TaskManager.Task::(_decref) Task=`7cabc832-0aad-4604-94d6-27e871a078ca`::ref 0 aborting False
>Thread-127744::DEBUG::2012-12-08 14:01:09,094::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127744::DEBUG::2012-12-08 14:01:09,095::task::588::TaskManager.Task::(_updateState) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::moving from state init -> state preparing
>Thread-127744::INFO::2012-12-08 14:01:09,096::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127744::DEBUG::2012-12-08 14:01:09,097::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`293f066e-9477-4c1e-a73d-b97b923da816`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127744::DEBUG::2012-12-08 14:01:09,098::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127744::DEBUG::2012-12-08 14:01:09,098::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127744::DEBUG::2012-12-08 14:01:09,099::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`293f066e-9477-4c1e-a73d-b97b923da816`::Granted request
>Thread-127744::DEBUG::2012-12-08 14:01:09,100::task::817::TaskManager.Task::(resourceAcquired) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127744::DEBUG::2012-12-08 14:01:09,100::task::978::TaskManager.Task::(_decref) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::ref 1 aborting False
>Thread-127744::INFO::2012-12-08 14:01:09,102::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127744::DEBUG::2012-12-08 14:01:09,103::task::1172::TaskManager.Task::(prepare) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127744::DEBUG::2012-12-08 14:01:09,103::task::588::TaskManager.Task::(_updateState) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::moving from state preparing -> state finished
>Thread-127744::DEBUG::2012-12-08 14:01:09,103::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127744::DEBUG::2012-12-08 14:01:09,104::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127744::DEBUG::2012-12-08 14:01:09,104::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127744::DEBUG::2012-12-08 14:01:09,104::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127744::DEBUG::2012-12-08 14:01:09,105::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127744::DEBUG::2012-12-08 14:01:09,105::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127744::DEBUG::2012-12-08 14:01:09,106::task::978::TaskManager.Task::(_decref) Task=`3646e275-d387-4871-91e5-78390cf65ad2`::ref 0 aborting False
>Thread-127745::DEBUG::2012-12-08 14:01:09,458::task::588::TaskManager.Task::(_updateState) Task=`0e829368-41d4-4ded-83cd-458c6fb0f8cc`::moving from state init -> state preparing
>Thread-127745::INFO::2012-12-08 14:01:09,458::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127745::INFO::2012-12-08 14:01:09,459::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153880119324', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00154781341553', 'lastCheck': '4.5', 'code': 0, 'valid': True}}
>Thread-127745::DEBUG::2012-12-08 14:01:09,459::task::1172::TaskManager.Task::(prepare) Task=`0e829368-41d4-4ded-83cd-458c6fb0f8cc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153880119324', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00154781341553', 'lastCheck': '4.5', 'code': 0, 'valid': True}}
>Thread-127745::DEBUG::2012-12-08 14:01:09,460::task::588::TaskManager.Task::(_updateState) Task=`0e829368-41d4-4ded-83cd-458c6fb0f8cc`::moving from state preparing -> state finished
>Thread-127745::DEBUG::2012-12-08 14:01:09,460::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127745::DEBUG::2012-12-08 14:01:09,460::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127745::DEBUG::2012-12-08 14:01:09,461::task::978::TaskManager.Task::(_decref) Task=`0e829368-41d4-4ded-83cd-458c6fb0f8cc`::ref 0 aborting False
>Thread-127746::DEBUG::2012-12-08 14:01:09,490::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127746::DEBUG::2012-12-08 14:01:09,491::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:01:10,500::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:10,554::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280949 s, 36.4 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:12,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:12,607::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021042 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:14,614::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:14,661::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207322 s, 49.4 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:15,988::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:16,009::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00130775 s, 3.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:16,668::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:16,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027998 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:18,729::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:18,777::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222805 s, 46.0 MB/s\n'; <rc> = 0
>Thread-127751::DEBUG::2012-12-08 14:01:19,855::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127751::DEBUG::2012-12-08 14:01:19,856::task::588::TaskManager.Task::(_updateState) Task=`db2600c7-8d30-43f5-be18-de701cb29a5d`::moving from state init -> state preparing
>Thread-127751::INFO::2012-12-08 14:01:19,857::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127751::INFO::2012-12-08 14:01:19,857::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127751::DEBUG::2012-12-08 14:01:19,858::task::1172::TaskManager.Task::(prepare) Task=`db2600c7-8d30-43f5-be18-de701cb29a5d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127751::DEBUG::2012-12-08 14:01:19,858::task::588::TaskManager.Task::(_updateState) Task=`db2600c7-8d30-43f5-be18-de701cb29a5d`::moving from state preparing -> state finished
>Thread-127751::DEBUG::2012-12-08 14:01:19,859::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127751::DEBUG::2012-12-08 14:01:19,859::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127751::DEBUG::2012-12-08 14:01:19,861::task::978::TaskManager.Task::(_decref) Task=`db2600c7-8d30-43f5-be18-de701cb29a5d`::ref 0 aborting False
>Thread-127752::DEBUG::2012-12-08 14:01:19,879::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127752::DEBUG::2012-12-08 14:01:19,880::task::588::TaskManager.Task::(_updateState) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::moving from state init -> state preparing
>Thread-127752::INFO::2012-12-08 14:01:19,881::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127752::DEBUG::2012-12-08 14:01:19,881::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b1fb11b0-e36b-4a07-bc5f-2c9d02ced61b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127752::DEBUG::2012-12-08 14:01:19,882::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127752::DEBUG::2012-12-08 14:01:19,883::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127752::DEBUG::2012-12-08 14:01:19,883::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b1fb11b0-e36b-4a07-bc5f-2c9d02ced61b`::Granted request
>Thread-127752::DEBUG::2012-12-08 14:01:19,884::task::817::TaskManager.Task::(resourceAcquired) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127752::DEBUG::2012-12-08 14:01:19,884::task::978::TaskManager.Task::(_decref) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::ref 1 aborting False
>Thread-127752::INFO::2012-12-08 14:01:19,886::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127752::DEBUG::2012-12-08 14:01:19,886::task::1172::TaskManager.Task::(prepare) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127752::DEBUG::2012-12-08 14:01:19,887::task::588::TaskManager.Task::(_updateState) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::moving from state preparing -> state finished
>Thread-127752::DEBUG::2012-12-08 14:01:19,887::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127752::DEBUG::2012-12-08 14:01:19,887::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127752::DEBUG::2012-12-08 14:01:19,888::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127752::DEBUG::2012-12-08 14:01:19,888::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127752::DEBUG::2012-12-08 14:01:19,889::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127752::DEBUG::2012-12-08 14:01:19,889::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127752::DEBUG::2012-12-08 14:01:19,890::task::978::TaskManager.Task::(_decref) Task=`83a6810d-7b29-4e5d-a18e-f9d662062333`::ref 0 aborting False
>Thread-127753::DEBUG::2012-12-08 14:01:20,189::task::588::TaskManager.Task::(_updateState) Task=`66037612-067b-4c1d-94c9-d04a2f614e1e`::moving from state init -> state preparing
>Thread-127753::INFO::2012-12-08 14:01:20,189::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127753::INFO::2012-12-08 14:01:20,189::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0222270488739', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00331592559814', 'lastCheck': '5.3', 'code': 0, 'valid': True}}
>Thread-127753::DEBUG::2012-12-08 14:01:20,189::task::1172::TaskManager.Task::(prepare) Task=`66037612-067b-4c1d-94c9-d04a2f614e1e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0222270488739', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00331592559814', 'lastCheck': '5.3', 'code': 0, 'valid': True}}
>Thread-127753::DEBUG::2012-12-08 14:01:20,190::task::588::TaskManager.Task::(_updateState) Task=`66037612-067b-4c1d-94c9-d04a2f614e1e`::moving from state preparing -> state finished
>Thread-127753::DEBUG::2012-12-08 14:01:20,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127753::DEBUG::2012-12-08 14:01:20,190::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127753::DEBUG::2012-12-08 14:01:20,190::task::978::TaskManager.Task::(_decref) Task=`66037612-067b-4c1d-94c9-d04a2f614e1e`::ref 0 aborting False
>Thread-127754::DEBUG::2012-12-08 14:01:20,204::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127754::DEBUG::2012-12-08 14:01:20,205::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:01:20,784::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:20,837::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0253194 s, 40.4 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:22,845::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:22,885::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166158 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:24,892::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:24,958::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261868 s, 39.1 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:26,014::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:26,030::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00116008 s, 3.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:26,965::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:27,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180815 s, 56.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:29,013::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:29,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254744 s, 40.2 MB/s\n'; <rc> = 0
>Thread-127759::DEBUG::2012-12-08 14:01:30,100::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127759::DEBUG::2012-12-08 14:01:30,102::task::588::TaskManager.Task::(_updateState) Task=`a304f8c3-00e4-429d-a874-ffb11704a0af`::moving from state init -> state preparing
>Thread-127759::INFO::2012-12-08 14:01:30,102::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127759::INFO::2012-12-08 14:01:30,103::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127759::DEBUG::2012-12-08 14:01:30,103::task::1172::TaskManager.Task::(prepare) Task=`a304f8c3-00e4-429d-a874-ffb11704a0af`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127759::DEBUG::2012-12-08 14:01:30,104::task::588::TaskManager.Task::(_updateState) Task=`a304f8c3-00e4-429d-a874-ffb11704a0af`::moving from state preparing -> state finished
>Thread-127759::DEBUG::2012-12-08 14:01:30,104::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127759::DEBUG::2012-12-08 14:01:30,105::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127759::DEBUG::2012-12-08 14:01:30,105::task::978::TaskManager.Task::(_decref) Task=`a304f8c3-00e4-429d-a874-ffb11704a0af`::ref 0 aborting False
>Thread-127760::DEBUG::2012-12-08 14:01:30,122::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127760::DEBUG::2012-12-08 14:01:30,123::task::588::TaskManager.Task::(_updateState) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::moving from state init -> state preparing
>Thread-127760::INFO::2012-12-08 14:01:30,124::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127760::DEBUG::2012-12-08 14:01:30,124::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de26689a-237d-4e68-bd76-9534d2b64b7f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127760::DEBUG::2012-12-08 14:01:30,125::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127760::DEBUG::2012-12-08 14:01:30,125::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127760::DEBUG::2012-12-08 14:01:30,126::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de26689a-237d-4e68-bd76-9534d2b64b7f`::Granted request
>Thread-127760::DEBUG::2012-12-08 14:01:30,126::task::817::TaskManager.Task::(resourceAcquired) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127760::DEBUG::2012-12-08 14:01:30,127::task::978::TaskManager.Task::(_decref) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::ref 1 aborting False
>Thread-127760::INFO::2012-12-08 14:01:30,128::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127760::DEBUG::2012-12-08 14:01:30,128::task::1172::TaskManager.Task::(prepare) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127760::DEBUG::2012-12-08 14:01:30,128::task::588::TaskManager.Task::(_updateState) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::moving from state preparing -> state finished
>Thread-127760::DEBUG::2012-12-08 14:01:30,128::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127760::DEBUG::2012-12-08 14:01:30,129::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127760::DEBUG::2012-12-08 14:01:30,129::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127760::DEBUG::2012-12-08 14:01:30,129::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127760::DEBUG::2012-12-08 14:01:30,129::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127760::DEBUG::2012-12-08 14:01:30,129::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127760::DEBUG::2012-12-08 14:01:30,130::task::978::TaskManager.Task::(_decref) Task=`43cc8165-2276-41d2-91b8-27363dde864b`::ref 0 aborting False
>Thread-127761::DEBUG::2012-12-08 14:01:30,560::task::588::TaskManager.Task::(_updateState) Task=`9cd173c3-a0d2-42e3-8d3d-889452ca0244`::moving from state init -> state preparing
>Thread-127761::INFO::2012-12-08 14:01:30,561::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127761::INFO::2012-12-08 14:01:30,562::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162518024445', 'lastCheck': '4.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00209903717041', 'lastCheck': '5.6', 'code': 0, 'valid': True}}
>Thread-127761::DEBUG::2012-12-08 14:01:30,562::task::1172::TaskManager.Task::(prepare) Task=`9cd173c3-a0d2-42e3-8d3d-889452ca0244`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162518024445', 'lastCheck': '4.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00209903717041', 'lastCheck': '5.6', 'code': 0, 'valid': True}}
>Thread-127761::DEBUG::2012-12-08 14:01:30,563::task::588::TaskManager.Task::(_updateState) Task=`9cd173c3-a0d2-42e3-8d3d-889452ca0244`::moving from state preparing -> state finished
>Thread-127761::DEBUG::2012-12-08 14:01:30,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127761::DEBUG::2012-12-08 14:01:30,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127761::DEBUG::2012-12-08 14:01:30,565::task::978::TaskManager.Task::(_decref) Task=`9cd173c3-a0d2-42e3-8d3d-889452ca0244`::ref 0 aborting False
>Thread-127762::DEBUG::2012-12-08 14:01:30,591::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127762::DEBUG::2012-12-08 14:01:30,593::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:01:31,071::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:31,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193461 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:33,124::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:33,175::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248476 s, 41.2 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:35,181::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:35,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214317 s, 47.8 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:36,033::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:36,049::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000901667 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:37,234::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:37,276::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168716 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:39,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:39,336::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260695 s, 39.3 MB/s\n'; <rc> = 0
>Thread-127767::DEBUG::2012-12-08 14:01:40,666::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127767::DEBUG::2012-12-08 14:01:40,667::task::588::TaskManager.Task::(_updateState) Task=`f656cad9-77ec-4471-8bd7-476e38c77fe6`::moving from state init -> state preparing
>Thread-127767::INFO::2012-12-08 14:01:40,667::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127767::INFO::2012-12-08 14:01:40,668::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127767::DEBUG::2012-12-08 14:01:40,668::task::1172::TaskManager.Task::(prepare) Task=`f656cad9-77ec-4471-8bd7-476e38c77fe6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127767::DEBUG::2012-12-08 14:01:40,669::task::588::TaskManager.Task::(_updateState) Task=`f656cad9-77ec-4471-8bd7-476e38c77fe6`::moving from state preparing -> state finished
>Thread-127767::DEBUG::2012-12-08 14:01:40,669::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127767::DEBUG::2012-12-08 14:01:40,670::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127767::DEBUG::2012-12-08 14:01:40,670::task::978::TaskManager.Task::(_decref) Task=`f656cad9-77ec-4471-8bd7-476e38c77fe6`::ref 0 aborting False
>Thread-127768::DEBUG::2012-12-08 14:01:40,691::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127768::DEBUG::2012-12-08 14:01:40,692::task::588::TaskManager.Task::(_updateState) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::moving from state init -> state preparing
>Thread-127768::INFO::2012-12-08 14:01:40,693::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127768::DEBUG::2012-12-08 14:01:40,694::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`618c118f-915c-4475-8646-90a06c183376`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127768::DEBUG::2012-12-08 14:01:40,694::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127768::DEBUG::2012-12-08 14:01:40,695::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127768::DEBUG::2012-12-08 14:01:40,695::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`618c118f-915c-4475-8646-90a06c183376`::Granted request
>Thread-127768::DEBUG::2012-12-08 14:01:40,696::task::817::TaskManager.Task::(resourceAcquired) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127768::DEBUG::2012-12-08 14:01:40,696::task::978::TaskManager.Task::(_decref) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::ref 1 aborting False
>Thread-127768::INFO::2012-12-08 14:01:40,698::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127768::DEBUG::2012-12-08 14:01:40,698::task::1172::TaskManager.Task::(prepare) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127768::DEBUG::2012-12-08 14:01:40,699::task::588::TaskManager.Task::(_updateState) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::moving from state preparing -> state finished
>Thread-127768::DEBUG::2012-12-08 14:01:40,699::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127768::DEBUG::2012-12-08 14:01:40,699::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127768::DEBUG::2012-12-08 14:01:40,699::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127768::DEBUG::2012-12-08 14:01:40,699::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127768::DEBUG::2012-12-08 14:01:40,700::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127768::DEBUG::2012-12-08 14:01:40,700::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127768::DEBUG::2012-12-08 14:01:40,700::task::978::TaskManager.Task::(_decref) Task=`d6cd0009-813e-4c14-ac27-988492bec91e`::ref 0 aborting False
>Thread-127769::DEBUG::2012-12-08 14:01:40,904::task::588::TaskManager.Task::(_updateState) Task=`a44cca1e-eb6d-4744-b6a9-b379189c5427`::moving from state init -> state preparing
>Thread-127769::INFO::2012-12-08 14:01:40,904::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127769::INFO::2012-12-08 14:01:40,904::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167109966278', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00262904167175', 'lastCheck': '5.9', 'code': 0, 'valid': True}}
>Thread-127769::DEBUG::2012-12-08 14:01:40,904::task::1172::TaskManager.Task::(prepare) Task=`a44cca1e-eb6d-4744-b6a9-b379189c5427`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167109966278', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00262904167175', 'lastCheck': '5.9', 'code': 0, 'valid': True}}
>Thread-127769::DEBUG::2012-12-08 14:01:40,905::task::588::TaskManager.Task::(_updateState) Task=`a44cca1e-eb6d-4744-b6a9-b379189c5427`::moving from state preparing -> state finished
>Thread-127769::DEBUG::2012-12-08 14:01:40,905::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127769::DEBUG::2012-12-08 14:01:40,905::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127769::DEBUG::2012-12-08 14:01:40,905::task::978::TaskManager.Task::(_decref) Task=`a44cca1e-eb6d-4744-b6a9-b379189c5427`::ref 0 aborting False
>Thread-127770::DEBUG::2012-12-08 14:01:40,930::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127770::DEBUG::2012-12-08 14:01:40,931::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:01:41,343::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:41,395::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262118 s, 39.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:43,400::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:43,455::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278719 s, 36.7 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:45,462::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:45,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265686 s, 38.5 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:46,053::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:46,068::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000744884 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:47,522::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:47,566::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194326 s, 52.7 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:49,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:49,624::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254894 s, 40.2 MB/s\n'; <rc> = 0
>Thread-127775::DEBUG::2012-12-08 14:01:50,924::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127775::DEBUG::2012-12-08 14:01:50,924::task::588::TaskManager.Task::(_updateState) Task=`3981c5f5-cd38-4da4-bd39-f8a94c7b5e07`::moving from state init -> state preparing
>Thread-127775::INFO::2012-12-08 14:01:50,925::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127775::INFO::2012-12-08 14:01:50,925::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127775::DEBUG::2012-12-08 14:01:50,925::task::1172::TaskManager.Task::(prepare) Task=`3981c5f5-cd38-4da4-bd39-f8a94c7b5e07`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127775::DEBUG::2012-12-08 14:01:50,925::task::588::TaskManager.Task::(_updateState) Task=`3981c5f5-cd38-4da4-bd39-f8a94c7b5e07`::moving from state preparing -> state finished
>Thread-127775::DEBUG::2012-12-08 14:01:50,926::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127775::DEBUG::2012-12-08 14:01:50,926::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127775::DEBUG::2012-12-08 14:01:50,926::task::978::TaskManager.Task::(_decref) Task=`3981c5f5-cd38-4da4-bd39-f8a94c7b5e07`::ref 0 aborting False
>Thread-127776::DEBUG::2012-12-08 14:01:50,938::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127776::DEBUG::2012-12-08 14:01:50,938::task::588::TaskManager.Task::(_updateState) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::moving from state init -> state preparing
>Thread-127776::INFO::2012-12-08 14:01:50,939::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127776::DEBUG::2012-12-08 14:01:50,939::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`32424070-9be9-436b-b0a8-90820afdee77`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127776::DEBUG::2012-12-08 14:01:50,939::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127776::DEBUG::2012-12-08 14:01:50,939::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127776::DEBUG::2012-12-08 14:01:50,940::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`32424070-9be9-436b-b0a8-90820afdee77`::Granted request
>Thread-127776::DEBUG::2012-12-08 14:01:50,940::task::817::TaskManager.Task::(resourceAcquired) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127776::DEBUG::2012-12-08 14:01:50,940::task::978::TaskManager.Task::(_decref) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::ref 1 aborting False
>Thread-127776::INFO::2012-12-08 14:01:50,941::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127776::DEBUG::2012-12-08 14:01:50,941::task::1172::TaskManager.Task::(prepare) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127776::DEBUG::2012-12-08 14:01:50,942::task::588::TaskManager.Task::(_updateState) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::moving from state preparing -> state finished
>Thread-127776::DEBUG::2012-12-08 14:01:50,942::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127776::DEBUG::2012-12-08 14:01:50,942::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127776::DEBUG::2012-12-08 14:01:50,942::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127776::DEBUG::2012-12-08 14:01:50,942::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127776::DEBUG::2012-12-08 14:01:50,943::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127776::DEBUG::2012-12-08 14:01:50,943::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127776::DEBUG::2012-12-08 14:01:50,943::task::978::TaskManager.Task::(_decref) Task=`5f23a094-e9b5-432b-9991-9e3dc00bb454`::ref 0 aborting False
>Thread-127777::DEBUG::2012-12-08 14:01:51,173::task::588::TaskManager.Task::(_updateState) Task=`c929f3e6-eb20-40ab-93ab-ce1bebabf75d`::moving from state init -> state preparing
>Thread-127777::INFO::2012-12-08 14:01:51,173::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127777::INFO::2012-12-08 14:01:51,173::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154979228973', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00226211547852', 'lastCheck': '6.2', 'code': 0, 'valid': True}}
>Thread-127777::DEBUG::2012-12-08 14:01:51,174::task::1172::TaskManager.Task::(prepare) Task=`c929f3e6-eb20-40ab-93ab-ce1bebabf75d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154979228973', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00226211547852', 'lastCheck': '6.2', 'code': 0, 'valid': True}}
>Thread-127777::DEBUG::2012-12-08 14:01:51,174::task::588::TaskManager.Task::(_updateState) Task=`c929f3e6-eb20-40ab-93ab-ce1bebabf75d`::moving from state preparing -> state finished
>Thread-127777::DEBUG::2012-12-08 14:01:51,174::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127777::DEBUG::2012-12-08 14:01:51,174::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127777::DEBUG::2012-12-08 14:01:51,174::task::978::TaskManager.Task::(_decref) Task=`c929f3e6-eb20-40ab-93ab-ce1bebabf75d`::ref 0 aborting False
>Thread-127778::DEBUG::2012-12-08 14:01:51,189::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127778::DEBUG::2012-12-08 14:01:51,190::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Thread-62648::DEBUG::2012-12-08 14:01:51,209::task::588::TaskManager.Task::(_updateState) Task=`b7068efa-6c4c-4d96-8566-912eb9b0e088`::moving from state init -> state preparing
>Thread-62648::INFO::2012-12-08 14:01:51,210::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', imgUUID='e45efe8c-3fea-4386-a7ca-685c45e98500', volUUID='3a5c86c9-56f1-4f87-9288-70bd0ab3e461', options=None)
>Thread-62648::INFO::2012-12-08 14:01:51,210::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-62648::DEBUG::2012-12-08 14:01:51,210::task::1172::TaskManager.Task::(prepare) Task=`b7068efa-6c4c-4d96-8566-912eb9b0e088`::finished: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-62648::DEBUG::2012-12-08 14:01:51,210::task::588::TaskManager.Task::(_updateState) Task=`b7068efa-6c4c-4d96-8566-912eb9b0e088`::moving from state preparing -> state finished
>Thread-62648::DEBUG::2012-12-08 14:01:51,211::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-62648::DEBUG::2012-12-08 14:01:51,211::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-62648::DEBUG::2012-12-08 14:01:51,211::task::978::TaskManager.Task::(_decref) Task=`b7068efa-6c4c-4d96-8566-912eb9b0e088`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:01:51,630::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:51,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207607 s, 49.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:53,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:53,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175097 s, 58.5 MB/s\n'; <rc> = 0
>Thread-1168::DEBUG::2012-12-08 14:01:54,039::task::588::TaskManager.Task::(_updateState) Task=`4fb835bc-ca42-48cb-b8fc-3c1bcda51af5`::moving from state init -> state preparing
>Thread-1168::INFO::2012-12-08 14:01:54,040::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', imgUUID='3a661f02-41b6-42ff-aa89-418f0f7f54cb', volUUID='38c6b100-31f8-4914-a760-eb1c5e15cce4', options=None)
>Thread-1168::INFO::2012-12-08 14:01:54,041::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '10737418240', 'apparentsize': '10737418240'}
>Thread-1168::DEBUG::2012-12-08 14:01:54,041::task::1172::TaskManager.Task::(prepare) Task=`4fb835bc-ca42-48cb-b8fc-3c1bcda51af5`::finished: {'truesize': '10737418240', 'apparentsize': '10737418240'}
>Thread-1168::DEBUG::2012-12-08 14:01:54,041::task::588::TaskManager.Task::(_updateState) Task=`4fb835bc-ca42-48cb-b8fc-3c1bcda51af5`::moving from state preparing -> state finished
>Thread-1168::DEBUG::2012-12-08 14:01:54,042::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1168::DEBUG::2012-12-08 14:01:54,042::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1168::DEBUG::2012-12-08 14:01:54,042::task::978::TaskManager.Task::(_decref) Task=`4fb835bc-ca42-48cb-b8fc-3c1bcda51af5`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:01:55,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:55,771::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166555 s, 61.5 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:01:56,071::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:01:56,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000993373 s, 4.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:57,778::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:57,831::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282443 s, 36.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:01:59,838::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:01:59,879::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170519 s, 60.1 MB/s\n'; <rc> = 0
>Thread-127783::DEBUG::2012-12-08 14:02:01,471::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127783::DEBUG::2012-12-08 14:02:01,472::task::588::TaskManager.Task::(_updateState) Task=`0136872a-9fa7-45df-8ef9-0913681e8b60`::moving from state init -> state preparing
>Thread-127783::INFO::2012-12-08 14:02:01,473::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127783::INFO::2012-12-08 14:02:01,473::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127783::DEBUG::2012-12-08 14:02:01,474::task::1172::TaskManager.Task::(prepare) Task=`0136872a-9fa7-45df-8ef9-0913681e8b60`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127783::DEBUG::2012-12-08 14:02:01,474::task::588::TaskManager.Task::(_updateState) Task=`0136872a-9fa7-45df-8ef9-0913681e8b60`::moving from state preparing -> state finished
>Thread-127783::DEBUG::2012-12-08 14:02:01,475::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127783::DEBUG::2012-12-08 14:02:01,475::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127783::DEBUG::2012-12-08 14:02:01,475::task::978::TaskManager.Task::(_decref) Task=`0136872a-9fa7-45df-8ef9-0913681e8b60`::ref 0 aborting False
>Thread-127784::DEBUG::2012-12-08 14:02:01,492::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127784::DEBUG::2012-12-08 14:02:01,492::task::588::TaskManager.Task::(_updateState) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::moving from state init -> state preparing
>Thread-127784::INFO::2012-12-08 14:02:01,493::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127784::DEBUG::2012-12-08 14:02:01,493::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ca02daba-0aa1-4714-8875-7c78f726fb69`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127784::DEBUG::2012-12-08 14:02:01,494::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127784::DEBUG::2012-12-08 14:02:01,494::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127784::DEBUG::2012-12-08 14:02:01,495::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ca02daba-0aa1-4714-8875-7c78f726fb69`::Granted request
>Thread-127784::DEBUG::2012-12-08 14:02:01,495::task::817::TaskManager.Task::(resourceAcquired) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127784::DEBUG::2012-12-08 14:02:01,496::task::978::TaskManager.Task::(_decref) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::ref 1 aborting False
>Thread-127784::INFO::2012-12-08 14:02:01,497::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127784::DEBUG::2012-12-08 14:02:01,498::task::1172::TaskManager.Task::(prepare) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127784::DEBUG::2012-12-08 14:02:01,498::task::588::TaskManager.Task::(_updateState) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::moving from state preparing -> state finished
>Thread-127784::DEBUG::2012-12-08 14:02:01,498::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127784::DEBUG::2012-12-08 14:02:01,498::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127784::DEBUG::2012-12-08 14:02:01,499::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127784::DEBUG::2012-12-08 14:02:01,499::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127784::DEBUG::2012-12-08 14:02:01,499::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127784::DEBUG::2012-12-08 14:02:01,499::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127784::DEBUG::2012-12-08 14:02:01,499::task::978::TaskManager.Task::(_decref) Task=`eff0c270-33a9-4afb-8c33-f474e2511675`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:02:01,886::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:01,932::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195189 s, 52.5 MB/s\n'; <rc> = 0
>Thread-127785::DEBUG::2012-12-08 14:02:02,058::task::588::TaskManager.Task::(_updateState) Task=`eec8ef7d-7ff8-45bd-8214-8c3bef10d8b5`::moving from state init -> state preparing
>Thread-127785::INFO::2012-12-08 14:02:02,058::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127785::INFO::2012-12-08 14:02:02,059::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168120861053', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00178694725037', 'lastCheck': '7.1', 'code': 0, 'valid': True}}
>Thread-127785::DEBUG::2012-12-08 14:02:02,059::task::1172::TaskManager.Task::(prepare) Task=`eec8ef7d-7ff8-45bd-8214-8c3bef10d8b5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168120861053', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00178694725037', 'lastCheck': '7.1', 'code': 0, 'valid': True}}
>Thread-127785::DEBUG::2012-12-08 14:02:02,059::task::588::TaskManager.Task::(_updateState) Task=`eec8ef7d-7ff8-45bd-8214-8c3bef10d8b5`::moving from state preparing -> state finished
>Thread-127785::DEBUG::2012-12-08 14:02:02,059::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127785::DEBUG::2012-12-08 14:02:02,059::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127785::DEBUG::2012-12-08 14:02:02,060::task::978::TaskManager.Task::(_decref) Task=`eec8ef7d-7ff8-45bd-8214-8c3bef10d8b5`::ref 0 aborting False
>Thread-127786::DEBUG::2012-12-08 14:02:02,076::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127786::DEBUG::2012-12-08 14:02:02,076::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:02:03,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:03,987::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021514 s, 47.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:05,995::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:06,040::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187252 s, 54.7 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:02:06,091::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:06,100::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000555795 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:08,047::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:08,089::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164898 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:10,097::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:10,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187029 s, 54.8 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:12,147::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:12,187::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165827 s, 61.8 MB/s\n'; <rc> = 0
>Thread-127791::DEBUG::2012-12-08 14:02:12,523::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127791::DEBUG::2012-12-08 14:02:12,524::task::588::TaskManager.Task::(_updateState) Task=`46fda5b3-6488-46f8-a600-73b5dfa5e60d`::moving from state init -> state preparing
>Thread-127791::INFO::2012-12-08 14:02:12,524::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127791::INFO::2012-12-08 14:02:12,524::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127791::DEBUG::2012-12-08 14:02:12,524::task::1172::TaskManager.Task::(prepare) Task=`46fda5b3-6488-46f8-a600-73b5dfa5e60d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127791::DEBUG::2012-12-08 14:02:12,525::task::588::TaskManager.Task::(_updateState) Task=`46fda5b3-6488-46f8-a600-73b5dfa5e60d`::moving from state preparing -> state finished
>Thread-127791::DEBUG::2012-12-08 14:02:12,525::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127791::DEBUG::2012-12-08 14:02:12,525::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127791::DEBUG::2012-12-08 14:02:12,525::task::978::TaskManager.Task::(_decref) Task=`46fda5b3-6488-46f8-a600-73b5dfa5e60d`::ref 0 aborting False
>Thread-127792::DEBUG::2012-12-08 14:02:12,539::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127792::DEBUG::2012-12-08 14:02:12,540::task::588::TaskManager.Task::(_updateState) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::moving from state init -> state preparing
>Thread-127792::INFO::2012-12-08 14:02:12,541::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127792::DEBUG::2012-12-08 14:02:12,542::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76a96b66-47fc-4fe7-8c15-52e9539743e8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127792::DEBUG::2012-12-08 14:02:12,542::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127792::DEBUG::2012-12-08 14:02:12,543::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127792::DEBUG::2012-12-08 14:02:12,543::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76a96b66-47fc-4fe7-8c15-52e9539743e8`::Granted request
>Thread-127792::DEBUG::2012-12-08 14:02:12,544::task::817::TaskManager.Task::(resourceAcquired) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127792::DEBUG::2012-12-08 14:02:12,544::task::978::TaskManager.Task::(_decref) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::ref 1 aborting False
>Thread-127792::INFO::2012-12-08 14:02:12,546::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127792::DEBUG::2012-12-08 14:02:12,546::task::1172::TaskManager.Task::(prepare) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127792::DEBUG::2012-12-08 14:02:12,547::task::588::TaskManager.Task::(_updateState) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::moving from state preparing -> state finished
>Thread-127792::DEBUG::2012-12-08 14:02:12,547::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127792::DEBUG::2012-12-08 14:02:12,548::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127792::DEBUG::2012-12-08 14:02:12,548::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127792::DEBUG::2012-12-08 14:02:12,548::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127792::DEBUG::2012-12-08 14:02:12,548::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127792::DEBUG::2012-12-08 14:02:12,548::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127792::DEBUG::2012-12-08 14:02:12,549::task::978::TaskManager.Task::(_decref) Task=`cba67d7c-8d87-4135-86e9-642d39af9d93`::ref 0 aborting False
>Thread-127793::DEBUG::2012-12-08 14:02:13,225::task::588::TaskManager.Task::(_updateState) Task=`39dccf0c-5898-41eb-bc83-0e13b71476c3`::moving from state init -> state preparing
>Thread-127793::INFO::2012-12-08 14:02:13,226::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127793::INFO::2012-12-08 14:02:13,226::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00924301147461', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00158214569092', 'lastCheck': '8.3', 'code': 0, 'valid': True}}
>Thread-127793::DEBUG::2012-12-08 14:02:13,226::task::1172::TaskManager.Task::(prepare) Task=`39dccf0c-5898-41eb-bc83-0e13b71476c3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00924301147461', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00158214569092', 'lastCheck': '8.3', 'code': 0, 'valid': True}}
>Thread-127793::DEBUG::2012-12-08 14:02:13,227::task::588::TaskManager.Task::(_updateState) Task=`39dccf0c-5898-41eb-bc83-0e13b71476c3`::moving from state preparing -> state finished
>Thread-127793::DEBUG::2012-12-08 14:02:13,227::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127793::DEBUG::2012-12-08 14:02:13,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127793::DEBUG::2012-12-08 14:02:13,228::task::978::TaskManager.Task::(_decref) Task=`39dccf0c-5898-41eb-bc83-0e13b71476c3`::ref 0 aborting False
>Thread-127794::DEBUG::2012-12-08 14:02:13,245::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127794::DEBUG::2012-12-08 14:02:13,246::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:02:14,194::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:14,241::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195759 s, 52.3 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:02:16,103::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:16,115::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000480921 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:16,248::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:16,289::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173088 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:18,295::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:18,340::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170044 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:20,347::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:20,392::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184852 s, 55.4 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:22,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:22,446::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202683 s, 50.5 MB/s\n'; <rc> = 0
>Thread-127799::DEBUG::2012-12-08 14:02:23,721::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127799::DEBUG::2012-12-08 14:02:23,722::task::588::TaskManager.Task::(_updateState) Task=`246b36c1-6254-4e54-8869-7df3f4d87d25`::moving from state init -> state preparing
>Thread-127799::INFO::2012-12-08 14:02:23,722::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127799::INFO::2012-12-08 14:02:23,723::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127799::DEBUG::2012-12-08 14:02:23,723::task::1172::TaskManager.Task::(prepare) Task=`246b36c1-6254-4e54-8869-7df3f4d87d25`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127799::DEBUG::2012-12-08 14:02:23,723::task::588::TaskManager.Task::(_updateState) Task=`246b36c1-6254-4e54-8869-7df3f4d87d25`::moving from state preparing -> state finished
>Thread-127799::DEBUG::2012-12-08 14:02:23,723::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127799::DEBUG::2012-12-08 14:02:23,723::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127799::DEBUG::2012-12-08 14:02:23,724::task::978::TaskManager.Task::(_decref) Task=`246b36c1-6254-4e54-8869-7df3f4d87d25`::ref 0 aborting False
>Thread-127800::DEBUG::2012-12-08 14:02:23,736::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127800::DEBUG::2012-12-08 14:02:23,737::task::588::TaskManager.Task::(_updateState) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::moving from state init -> state preparing
>Thread-127800::INFO::2012-12-08 14:02:23,737::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127800::DEBUG::2012-12-08 14:02:23,738::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a7071751-e1d0-4370-b4b3-6cf0339541bf`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127800::DEBUG::2012-12-08 14:02:23,738::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127800::DEBUG::2012-12-08 14:02:23,739::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127800::DEBUG::2012-12-08 14:02:23,739::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a7071751-e1d0-4370-b4b3-6cf0339541bf`::Granted request
>Thread-127800::DEBUG::2012-12-08 14:02:23,755::task::817::TaskManager.Task::(resourceAcquired) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127800::DEBUG::2012-12-08 14:02:23,755::task::978::TaskManager.Task::(_decref) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::ref 1 aborting False
>Thread-127800::INFO::2012-12-08 14:02:23,756::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127800::DEBUG::2012-12-08 14:02:23,756::task::1172::TaskManager.Task::(prepare) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127800::DEBUG::2012-12-08 14:02:23,756::task::588::TaskManager.Task::(_updateState) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::moving from state preparing -> state finished
>Thread-127800::DEBUG::2012-12-08 14:02:23,757::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127800::DEBUG::2012-12-08 14:02:23,757::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127800::DEBUG::2012-12-08 14:02:23,757::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127800::DEBUG::2012-12-08 14:02:23,757::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127800::DEBUG::2012-12-08 14:02:23,758::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127800::DEBUG::2012-12-08 14:02:23,758::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127800::DEBUG::2012-12-08 14:02:23,758::task::978::TaskManager.Task::(_decref) Task=`bd29e187-dec5-4329-a46d-9234bbfc195a`::ref 0 aborting False
>Thread-127801::DEBUG::2012-12-08 14:02:24,288::task::588::TaskManager.Task::(_updateState) Task=`980e00d0-f85e-428d-8af9-ab085cdc30cf`::moving from state init -> state preparing
>Thread-127801::INFO::2012-12-08 14:02:24,288::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127801::INFO::2012-12-08 14:02:24,289::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127010345459', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00108790397644', 'lastCheck': '9.3', 'code': 0, 'valid': True}}
>Thread-127801::DEBUG::2012-12-08 14:02:24,289::task::1172::TaskManager.Task::(prepare) Task=`980e00d0-f85e-428d-8af9-ab085cdc30cf`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127010345459', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00108790397644', 'lastCheck': '9.3', 'code': 0, 'valid': True}}
>Thread-127801::DEBUG::2012-12-08 14:02:24,289::task::588::TaskManager.Task::(_updateState) Task=`980e00d0-f85e-428d-8af9-ab085cdc30cf`::moving from state preparing -> state finished
>Thread-127801::DEBUG::2012-12-08 14:02:24,289::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127801::DEBUG::2012-12-08 14:02:24,289::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127801::DEBUG::2012-12-08 14:02:24,290::task::978::TaskManager.Task::(_decref) Task=`980e00d0-f85e-428d-8af9-ab085cdc30cf`::ref 0 aborting False
>Thread-127802::DEBUG::2012-12-08 14:02:24,304::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127802::DEBUG::2012-12-08 14:02:24,305::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:02:24,453::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:24,493::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159983 s, 64.0 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:02:26,121::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:26,138::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000939568 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:26,499::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:26,541::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165462 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:28,548::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:28,597::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221771 s, 46.2 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:30,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:30,642::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169329 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:32,649::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:32,703::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285993 s, 35.8 MB/s\n'; <rc> = 0
>Thread-127807::DEBUG::2012-12-08 14:02:34,187::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127807::DEBUG::2012-12-08 14:02:34,187::task::588::TaskManager.Task::(_updateState) Task=`9aff2765-97e8-40e4-9fcd-b77e562e6a2e`::moving from state init -> state preparing
>Thread-127807::INFO::2012-12-08 14:02:34,188::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127807::INFO::2012-12-08 14:02:34,188::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127807::DEBUG::2012-12-08 14:02:34,188::task::1172::TaskManager.Task::(prepare) Task=`9aff2765-97e8-40e4-9fcd-b77e562e6a2e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127807::DEBUG::2012-12-08 14:02:34,188::task::588::TaskManager.Task::(_updateState) Task=`9aff2765-97e8-40e4-9fcd-b77e562e6a2e`::moving from state preparing -> state finished
>Thread-127807::DEBUG::2012-12-08 14:02:34,189::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127807::DEBUG::2012-12-08 14:02:34,189::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127807::DEBUG::2012-12-08 14:02:34,189::task::978::TaskManager.Task::(_decref) Task=`9aff2765-97e8-40e4-9fcd-b77e562e6a2e`::ref 0 aborting False
>Thread-127808::DEBUG::2012-12-08 14:02:34,204::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127808::DEBUG::2012-12-08 14:02:34,205::task::588::TaskManager.Task::(_updateState) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::moving from state init -> state preparing
>Thread-127808::INFO::2012-12-08 14:02:34,205::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127808::DEBUG::2012-12-08 14:02:34,205::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`54ef57e4-52b5-47c6-968c-9bdcedf65dfa`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127808::DEBUG::2012-12-08 14:02:34,205::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127808::DEBUG::2012-12-08 14:02:34,206::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127808::DEBUG::2012-12-08 14:02:34,206::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`54ef57e4-52b5-47c6-968c-9bdcedf65dfa`::Granted request
>Thread-127808::DEBUG::2012-12-08 14:02:34,206::task::817::TaskManager.Task::(resourceAcquired) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127808::DEBUG::2012-12-08 14:02:34,206::task::978::TaskManager.Task::(_decref) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::ref 1 aborting False
>Thread-127808::INFO::2012-12-08 14:02:34,207::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127808::DEBUG::2012-12-08 14:02:34,207::task::1172::TaskManager.Task::(prepare) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127808::DEBUG::2012-12-08 14:02:34,208::task::588::TaskManager.Task::(_updateState) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::moving from state preparing -> state finished
>Thread-127808::DEBUG::2012-12-08 14:02:34,208::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127808::DEBUG::2012-12-08 14:02:34,208::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127808::DEBUG::2012-12-08 14:02:34,208::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127808::DEBUG::2012-12-08 14:02:34,209::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127808::DEBUG::2012-12-08 14:02:34,209::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127808::DEBUG::2012-12-08 14:02:34,209::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127808::DEBUG::2012-12-08 14:02:34,209::task::978::TaskManager.Task::(_decref) Task=`636d8077-24c0-4094-92e4-d9b212effc99`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:02:34,710::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:34,759::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227196 s, 45.1 MB/s\n'; <rc> = 0
>Thread-127809::DEBUG::2012-12-08 14:02:34,861::task::588::TaskManager.Task::(_updateState) Task=`7e603ab6-de21-44e1-9143-293506c8dbb2`::moving from state init -> state preparing
>Thread-127809::INFO::2012-12-08 14:02:34,861::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127809::INFO::2012-12-08 14:02:34,862::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0179030895233', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00134420394897', 'lastCheck': '9.9', 'code': 0, 'valid': True}}
>Thread-127809::DEBUG::2012-12-08 14:02:34,862::task::1172::TaskManager.Task::(prepare) Task=`7e603ab6-de21-44e1-9143-293506c8dbb2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0179030895233', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00134420394897', 'lastCheck': '9.9', 'code': 0, 'valid': True}}
>Thread-127809::DEBUG::2012-12-08 14:02:34,862::task::588::TaskManager.Task::(_updateState) Task=`7e603ab6-de21-44e1-9143-293506c8dbb2`::moving from state preparing -> state finished
>Thread-127809::DEBUG::2012-12-08 14:02:34,862::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127809::DEBUG::2012-12-08 14:02:34,863::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127809::DEBUG::2012-12-08 14:02:34,863::task::978::TaskManager.Task::(_decref) Task=`7e603ab6-de21-44e1-9143-293506c8dbb2`::ref 0 aborting False
>Thread-127810::DEBUG::2012-12-08 14:02:34,877::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127810::DEBUG::2012-12-08 14:02:34,878::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Thread-37::DEBUG::2012-12-08 14:02:36,141::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:36,158::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000734403 s, 5.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:36,766::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:36,814::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023037 s, 44.5 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:38,821::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:38,864::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191176 s, 53.6 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:40,869::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:40,921::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233446 s, 43.9 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:42,926::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:42,965::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015801 s, 64.8 MB/s\n'; <rc> = 0
>Thread-127815::DEBUG::2012-12-08 14:02:44,832::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127815::DEBUG::2012-12-08 14:02:44,833::task::588::TaskManager.Task::(_updateState) Task=`a0a7feb9-5550-48fa-b460-f32a24d1065a`::moving from state init -> state preparing
>Thread-127815::INFO::2012-12-08 14:02:44,833::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127815::INFO::2012-12-08 14:02:44,833::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127815::DEBUG::2012-12-08 14:02:44,834::task::1172::TaskManager.Task::(prepare) Task=`a0a7feb9-5550-48fa-b460-f32a24d1065a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127815::DEBUG::2012-12-08 14:02:44,834::task::588::TaskManager.Task::(_updateState) Task=`a0a7feb9-5550-48fa-b460-f32a24d1065a`::moving from state preparing -> state finished
>Thread-127815::DEBUG::2012-12-08 14:02:44,834::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127815::DEBUG::2012-12-08 14:02:44,834::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127815::DEBUG::2012-12-08 14:02:44,834::task::978::TaskManager.Task::(_decref) Task=`a0a7feb9-5550-48fa-b460-f32a24d1065a`::ref 0 aborting False
>Thread-127816::DEBUG::2012-12-08 14:02:44,846::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127816::DEBUG::2012-12-08 14:02:44,847::task::588::TaskManager.Task::(_updateState) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::moving from state init -> state preparing
>Thread-127816::INFO::2012-12-08 14:02:44,847::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127816::DEBUG::2012-12-08 14:02:44,847::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`560edeca-2313-4e7f-ab49-f842cd1be3e9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127816::DEBUG::2012-12-08 14:02:44,847::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127816::DEBUG::2012-12-08 14:02:44,848::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127816::DEBUG::2012-12-08 14:02:44,848::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`560edeca-2313-4e7f-ab49-f842cd1be3e9`::Granted request
>Thread-127816::DEBUG::2012-12-08 14:02:44,848::task::817::TaskManager.Task::(resourceAcquired) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127816::DEBUG::2012-12-08 14:02:44,848::task::978::TaskManager.Task::(_decref) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::ref 1 aborting False
>Thread-127816::INFO::2012-12-08 14:02:44,849::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127816::DEBUG::2012-12-08 14:02:44,849::task::1172::TaskManager.Task::(prepare) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127816::DEBUG::2012-12-08 14:02:44,850::task::588::TaskManager.Task::(_updateState) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::moving from state preparing -> state finished
>Thread-127816::DEBUG::2012-12-08 14:02:44,850::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127816::DEBUG::2012-12-08 14:02:44,850::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127816::DEBUG::2012-12-08 14:02:44,850::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127816::DEBUG::2012-12-08 14:02:44,850::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127816::DEBUG::2012-12-08 14:02:44,851::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127816::DEBUG::2012-12-08 14:02:44,851::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127816::DEBUG::2012-12-08 14:02:44,851::task::978::TaskManager.Task::(_decref) Task=`e2e32fc4-7338-4f06-aef0-7efe574bbcfa`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:02:44,972::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:45,021::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204809 s, 50.0 MB/s\n'; <rc> = 0
>Thread-127817::DEBUG::2012-12-08 14:02:46,019::task::588::TaskManager.Task::(_updateState) Task=`beeda38b-471e-4d83-a1b2-c511e4f935b7`::moving from state init -> state preparing
>Thread-127817::INFO::2012-12-08 14:02:46,019::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127817::INFO::2012-12-08 14:02:46,019::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0180299282074', 'lastCheck': '9.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00192594528198', 'lastCheck': '1.0', 'code': 0, 'valid': True}}
>Thread-127817::DEBUG::2012-12-08 14:02:46,019::task::1172::TaskManager.Task::(prepare) Task=`beeda38b-471e-4d83-a1b2-c511e4f935b7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0180299282074', 'lastCheck': '9.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00192594528198', 'lastCheck': '1.0', 'code': 0, 'valid': True}}
>Thread-127817::DEBUG::2012-12-08 14:02:46,020::task::588::TaskManager.Task::(_updateState) Task=`beeda38b-471e-4d83-a1b2-c511e4f935b7`::moving from state preparing -> state finished
>Thread-127817::DEBUG::2012-12-08 14:02:46,020::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127817::DEBUG::2012-12-08 14:02:46,020::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127817::DEBUG::2012-12-08 14:02:46,020::task::978::TaskManager.Task::(_decref) Task=`beeda38b-471e-4d83-a1b2-c511e4f935b7`::ref 0 aborting False
>Thread-127818::DEBUG::2012-12-08 14:02:46,039::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127818::DEBUG::2012-12-08 14:02:46,039::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Thread-37::DEBUG::2012-12-08 14:02:46,162::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:46,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00101594 s, 4.0 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:47,027::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:47,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165218 s, 62.0 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:49,074::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:49,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190038 s, 53.9 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:51,127::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:51,172::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195624 s, 52.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:53,178::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:53,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172607 s, 59.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:55,224::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:55,265::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175342 s, 58.4 MB/s\n'; <rc> = 0
>Thread-1168::DEBUG::2012-12-08 14:02:55,432::task::588::TaskManager.Task::(_updateState) Task=`08594841-a28f-42ac-80e5-8ba818b8236d`::moving from state init -> state preparing
>Thread-1168::INFO::2012-12-08 14:02:55,433::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', imgUUID='3a661f02-41b6-42ff-aa89-418f0f7f54cb', volUUID='38c6b100-31f8-4914-a760-eb1c5e15cce4', options=None)
>Thread-1168::INFO::2012-12-08 14:02:55,433::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '10737418240', 'apparentsize': '10737418240'}
>Thread-1168::DEBUG::2012-12-08 14:02:55,433::task::1172::TaskManager.Task::(prepare) Task=`08594841-a28f-42ac-80e5-8ba818b8236d`::finished: {'truesize': '10737418240', 'apparentsize': '10737418240'}
>Thread-1168::DEBUG::2012-12-08 14:02:55,434::task::588::TaskManager.Task::(_updateState) Task=`08594841-a28f-42ac-80e5-8ba818b8236d`::moving from state preparing -> state finished
>Thread-1168::DEBUG::2012-12-08 14:02:55,434::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1168::DEBUG::2012-12-08 14:02:55,434::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1168::DEBUG::2012-12-08 14:02:55,434::task::978::TaskManager.Task::(_decref) Task=`08594841-a28f-42ac-80e5-8ba818b8236d`::ref 0 aborting False
>Thread-127823::DEBUG::2012-12-08 14:02:56,049::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127823::DEBUG::2012-12-08 14:02:56,050::task::588::TaskManager.Task::(_updateState) Task=`a139287a-fe8d-4fd4-ba36-d04e22727deb`::moving from state init -> state preparing
>Thread-127823::INFO::2012-12-08 14:02:56,050::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127823::INFO::2012-12-08 14:02:56,051::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127823::DEBUG::2012-12-08 14:02:56,051::task::1172::TaskManager.Task::(prepare) Task=`a139287a-fe8d-4fd4-ba36-d04e22727deb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127823::DEBUG::2012-12-08 14:02:56,051::task::588::TaskManager.Task::(_updateState) Task=`a139287a-fe8d-4fd4-ba36-d04e22727deb`::moving from state preparing -> state finished
>Thread-127823::DEBUG::2012-12-08 14:02:56,051::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127823::DEBUG::2012-12-08 14:02:56,051::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127823::DEBUG::2012-12-08 14:02:56,052::task::978::TaskManager.Task::(_decref) Task=`a139287a-fe8d-4fd4-ba36-d04e22727deb`::ref 0 aborting False
>Thread-127824::DEBUG::2012-12-08 14:02:56,069::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127824::DEBUG::2012-12-08 14:02:56,069::task::588::TaskManager.Task::(_updateState) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::moving from state init -> state preparing
>Thread-127824::INFO::2012-12-08 14:02:56,069::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127824::DEBUG::2012-12-08 14:02:56,070::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7a15d6d-79ca-4163-b1cf-846a583e4e71`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127824::DEBUG::2012-12-08 14:02:56,070::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127824::DEBUG::2012-12-08 14:02:56,070::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127824::DEBUG::2012-12-08 14:02:56,071::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7a15d6d-79ca-4163-b1cf-846a583e4e71`::Granted request
>Thread-127824::DEBUG::2012-12-08 14:02:56,071::task::817::TaskManager.Task::(resourceAcquired) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127824::DEBUG::2012-12-08 14:02:56,071::task::978::TaskManager.Task::(_decref) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::ref 1 aborting False
>Thread-127824::INFO::2012-12-08 14:02:56,072::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127824::DEBUG::2012-12-08 14:02:56,072::task::1172::TaskManager.Task::(prepare) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127824::DEBUG::2012-12-08 14:02:56,072::task::588::TaskManager.Task::(_updateState) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::moving from state preparing -> state finished
>Thread-127824::DEBUG::2012-12-08 14:02:56,072::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127824::DEBUG::2012-12-08 14:02:56,073::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127824::DEBUG::2012-12-08 14:02:56,073::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127824::DEBUG::2012-12-08 14:02:56,073::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127824::DEBUG::2012-12-08 14:02:56,073::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127824::DEBUG::2012-12-08 14:02:56,074::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127824::DEBUG::2012-12-08 14:02:56,074::task::978::TaskManager.Task::(_decref) Task=`4b66b12e-40b4-49cf-acfb-db02e31ca9b7`::ref 0 aborting False
>Thread-37::DEBUG::2012-12-08 14:02:56,194::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:02:56,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000689013 s, 5.9 MB/s\n'; <rc> = 0
>Thread-127825::DEBUG::2012-12-08 14:02:56,826::task::588::TaskManager.Task::(_updateState) Task=`5b737a90-307d-419e-8fbb-e4d3ceeb90aa`::moving from state init -> state preparing
>Thread-127825::INFO::2012-12-08 14:02:56,827::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127825::INFO::2012-12-08 14:02:56,827::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263321399689', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00243496894836', 'lastCheck': '1.8', 'code': 0, 'valid': True}}
>Thread-127825::DEBUG::2012-12-08 14:02:56,827::task::1172::TaskManager.Task::(prepare) Task=`5b737a90-307d-419e-8fbb-e4d3ceeb90aa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263321399689', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00243496894836', 'lastCheck': '1.8', 'code': 0, 'valid': True}}
>Thread-127825::DEBUG::2012-12-08 14:02:56,827::task::588::TaskManager.Task::(_updateState) Task=`5b737a90-307d-419e-8fbb-e4d3ceeb90aa`::moving from state preparing -> state finished
>Thread-127825::DEBUG::2012-12-08 14:02:56,827::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127825::DEBUG::2012-12-08 14:02:56,828::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127825::DEBUG::2012-12-08 14:02:56,828::task::978::TaskManager.Task::(_decref) Task=`5b737a90-307d-419e-8fbb-e4d3ceeb90aa`::ref 0 aborting False
>Thread-127826::DEBUG::2012-12-08 14:02:56,849::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127826::DEBUG::2012-12-08 14:02:56,849::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:02:57,272::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:57,316::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199742 s, 51.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:02:59,323::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:02:59,365::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168378 s, 60.8 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:01,372::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:01,414::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160153 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:03,421::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:03,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168686 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:05,469::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:05,515::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224417 s, 45.6 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:03:06,223::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:03:06,235::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000568606 s, 7.2 MB/s\n'; <rc> = 0
>Thread-127831::DEBUG::2012-12-08 14:03:07,483::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127831::DEBUG::2012-12-08 14:03:07,483::task::588::TaskManager.Task::(_updateState) Task=`6d9bc378-1f39-43af-987d-5a6d2cb31f7a`::moving from state init -> state preparing
>Thread-127831::INFO::2012-12-08 14:03:07,484::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127831::INFO::2012-12-08 14:03:07,484::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127831::DEBUG::2012-12-08 14:03:07,484::task::1172::TaskManager.Task::(prepare) Task=`6d9bc378-1f39-43af-987d-5a6d2cb31f7a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127831::DEBUG::2012-12-08 14:03:07,484::task::588::TaskManager.Task::(_updateState) Task=`6d9bc378-1f39-43af-987d-5a6d2cb31f7a`::moving from state preparing -> state finished
>Thread-127831::DEBUG::2012-12-08 14:03:07,484::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127831::DEBUG::2012-12-08 14:03:07,485::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127831::DEBUG::2012-12-08 14:03:07,485::task::978::TaskManager.Task::(_decref) Task=`6d9bc378-1f39-43af-987d-5a6d2cb31f7a`::ref 0 aborting False
>Thread-127832::DEBUG::2012-12-08 14:03:07,500::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127832::DEBUG::2012-12-08 14:03:07,500::task::588::TaskManager.Task::(_updateState) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::moving from state init -> state preparing
>Thread-127832::INFO::2012-12-08 14:03:07,501::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127832::DEBUG::2012-12-08 14:03:07,501::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7da4abf6-2178-4bcb-b78a-977923c69fc8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-127832::DEBUG::2012-12-08 14:03:07,501::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-127832::DEBUG::2012-12-08 14:03:07,501::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-127832::DEBUG::2012-12-08 14:03:07,502::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7da4abf6-2178-4bcb-b78a-977923c69fc8`::Granted request
>Thread-127832::DEBUG::2012-12-08 14:03:07,502::task::817::TaskManager.Task::(resourceAcquired) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-127832::DEBUG::2012-12-08 14:03:07,502::task::978::TaskManager.Task::(_decref) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::ref 1 aborting False
>Thread-127832::INFO::2012-12-08 14:03:07,503::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127832::DEBUG::2012-12-08 14:03:07,503::task::1172::TaskManager.Task::(prepare) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141/images/11111111-1111-1111-1111-111111111111', 'type': 'ISCSI', 'master_ver': 1, 'lver': 48}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'diskfree': '12327583744', 'alerts': [], 'disktotal': '26958888960'}}}
>Thread-127832::DEBUG::2012-12-08 14:03:07,503::task::588::TaskManager.Task::(_updateState) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::moving from state preparing -> state finished
>Thread-127832::DEBUG::2012-12-08 14:03:07,504::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-127832::DEBUG::2012-12-08 14:03:07,504::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127832::DEBUG::2012-12-08 14:03:07,504::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-127832::DEBUG::2012-12-08 14:03:07,504::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-127832::DEBUG::2012-12-08 14:03:07,505::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-127832::DEBUG::2012-12-08 14:03:07,505::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-127832::DEBUG::2012-12-08 14:03:07,505::task::978::TaskManager.Task::(_decref) Task=`b755054c-c40a-4e40-bb50-ac588d7aaf03`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:03:07,522::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:07,567::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201971 s, 50.7 MB/s\n'; <rc> = 0
>Thread-127833::DEBUG::2012-12-08 14:03:08,936::task::588::TaskManager.Task::(_updateState) Task=`77fe3334-cf24-4707-a9f7-df759316e676`::moving from state init -> state preparing
>Thread-127833::INFO::2012-12-08 14:03:08,936::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-127833::INFO::2012-12-08 14:03:08,937::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.012157201767', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00219798088074', 'lastCheck': '3.9', 'code': 0, 'valid': True}}
>Thread-127833::DEBUG::2012-12-08 14:03:08,937::task::1172::TaskManager.Task::(prepare) Task=`77fe3334-cf24-4707-a9f7-df759316e676`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.012157201767', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00219798088074', 'lastCheck': '3.9', 'code': 0, 'valid': True}}
>Thread-127833::DEBUG::2012-12-08 14:03:08,937::task::588::TaskManager.Task::(_updateState) Task=`77fe3334-cf24-4707-a9f7-df759316e676`::moving from state preparing -> state finished
>Thread-127833::DEBUG::2012-12-08 14:03:08,937::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127833::DEBUG::2012-12-08 14:03:08,937::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127833::DEBUG::2012-12-08 14:03:08,938::task::978::TaskManager.Task::(_decref) Task=`77fe3334-cf24-4707-a9f7-df759316e676`::ref 0 aborting False
>Thread-127834::DEBUG::2012-12-08 14:03:08,955::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`c204995f-589e-4110-97ef-76494505123a`::Disk hdc stats not available
>Thread-127834::DEBUG::2012-12-08 14:03:08,956::libvirtvm::267::vm.Vm::(_getDiskStats) vmId=`05bc5d3a-653b-440f-900c-922c99fd6be3`::Disk hdc stats not available
>Dummy-85::DEBUG::2012-12-08 14:03:09,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:09,612::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162377 s, 63.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:11,618::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:11,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0309142 s, 33.1 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:13,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:13,726::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017236 s, 59.4 MB/s\n'; <rc> = 0
>Thread-62648::DEBUG::2012-12-08 14:03:13,899::task::588::TaskManager.Task::(_updateState) Task=`0f8f2875-543e-4944-9b4e-2f6f88f5a813`::moving from state init -> state preparing
>Thread-62648::INFO::2012-12-08 14:03:13,899::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', imgUUID='e45efe8c-3fea-4386-a7ca-685c45e98500', volUUID='3a5c86c9-56f1-4f87-9288-70bd0ab3e461', options=None)
>Thread-62648::INFO::2012-12-08 14:03:13,900::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-62648::DEBUG::2012-12-08 14:03:13,900::task::1172::TaskManager.Task::(prepare) Task=`0f8f2875-543e-4944-9b4e-2f6f88f5a813`::finished: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-62648::DEBUG::2012-12-08 14:03:13,900::task::588::TaskManager.Task::(_updateState) Task=`0f8f2875-543e-4944-9b4e-2f6f88f5a813`::moving from state preparing -> state finished
>Thread-62648::DEBUG::2012-12-08 14:03:13,901::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-62648::DEBUG::2012-12-08 14:03:13,901::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-62648::DEBUG::2012-12-08 14:03:13,901::task::978::TaskManager.Task::(_decref) Task=`0f8f2875-543e-4944-9b4e-2f6f88f5a813`::ref 0 aborting False
>Dummy-85::DEBUG::2012-12-08 14:03:15,735::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:15,777::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175466 s, 58.4 MB/s\n'; <rc> = 0
>Thread-37::DEBUG::2012-12-08 14:03:16,238::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-37::DEBUG::2012-12-08 14:03:16,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000947417 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-85::DEBUG::2012-12-08 14:03:17,784::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-85::DEBUG::2012-12-08 14:03:17,824::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160523 s, 63.8 MB/s\n'; <rc> = 0
>Thread-127839::DEBUG::2012-12-08 14:03:19,093::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127839::DEBUG::2012-12-08 14:03:19,093::task::588::TaskManager.Task::(_updateState) Task=`815fce27-2aa8-4d31-85ae-c33c224ba7e0`::moving from state init -> state preparing
>Thread-127839::INFO::2012-12-08 14:03:19,094::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127839::INFO::2012-12-08 14:03:19,095::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127839::DEBUG::2012-12-08 14:03:19,095::task::1172::TaskManager.Task::(prepare) Task=`815fce27-2aa8-4d31-85ae-c33c224ba7e0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 48}}
>Thread-127839::DEBUG::2012-12-08 14:03:19,096::task::588::TaskManager.Task::(_updateState) Task=`815fce27-2aa8-4d31-85ae-c33c224ba7e0`::moving from state preparing -> state finished
>Thread-127839::DEBUG::2012-12-08 14:03:19,096::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-127839::DEBUG::2012-12-08 14:03:19,096::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-127839::DEBUG::2012-12-08 14:03:19,097::task::978::TaskManager.Task::(_decref) Task=`815fce27-2aa8-4d31-85ae-c33c224ba7e0`::ref 0 aborting False
>Thread-127840::DEBUG::2012-12-08 14:03:19,111::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-127840::DEBUG::2012-12-08 14:03:19,112::task::588::TaskManager.Task::(_updateState) Task=`a829158b-9161-4905-a1a9-0378c928e71f`::moving from state init -> state preparing
>Thread-127840::INFO::2012-12-08 14:03:19,113::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-127840::DEBUG::2012-12-08 14:03:19,114::resourceManager::175::ReMainThread::INFO::2012-12-09 12:02:58,857::vdsm::70::vds::(run) I am the actual vdsm 4.9-44.0
>MainThread::DEBUG::2012-12-09 12:02:59,451::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage'
>MainThread::DEBUG::2012-12-09 12:02:59,451::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0
>MainThread::WARNING::2012-12-09 12:02:59,470::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/mnt already exists
>MainThread::DEBUG::2012-12-09 12:02:59,475::sp::363::Storage.StoragePool::(cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master` is not mounted, skipping
>MainThread::DEBUG::2012-12-09 12:02:59,510::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/cat /etc/multipath.conf' (cwd None)
>MainThread::DEBUG::2012-12-09 12:02:59,575::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>MainThread::DEBUG::2012-12-09 12:02:59,576::multipath::103::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving
>MainThread::DEBUG::2012-12-09 12:02:59,576::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None)
>MainThread::DEBUG::2012-12-09 12:02:59,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>MainThread::DEBUG::2012-12-09 12:02:59,677::hsm::407::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center'
>MainThread::DEBUG::2012-12-09 12:02:59,679::hsm::439::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt']
>MainThread::DEBUG::2012-12-09 12:02:59,679::hsm::440::Storage.HSM::(__cleanStorageRepository) Mount list: []
>MainThread::DEBUG::2012-12-09 12:02:59,679::hsm::442::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers
>MainThread::DEBUG::2012-12-09 12:02:59,684::hsm::485::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center'
>Thread-12::DEBUG::2012-12-09 12:02:59,686::lvm::319::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex
>MainThread::INFO::2012-12-09 12:02:59,686::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher...
>Thread-12::DEBUG::2012-12-09 12:02:59,689::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None)
>Thread-12::DEBUG::2012-12-09 12:02:59,784::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-12::DEBUG::2012-12-09 12:02:59,789::lvm::342::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex
>Thread-12::DEBUG::2012-12-09 12:02:59,789::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-12::DEBUG::2012-12-09 12:02:59,789::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None)
>MainThread::DEBUG::2012-12-09 12:02:59,825::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None)
>Thread-12::DEBUG::2012-12-09 12:02:59,902::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>Thread-12::DEBUG::2012-12-09 12:02:59,903::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-12::DEBUG::2012-12-09 12:02:59,903::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None)
>MainThread::DEBUG::2012-12-09 12:02:59,915::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>MainThread::INFO::2012-12-09 12:02:59,915::ksm::40::vds::(__init__) starting ksm monitor thread, ksm pid is 39
>KsmMonitor::DEBUG::2012-12-09 12:02:59,916::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/service ksmtuned start' (cwd None)
>MainThread::INFO::2012-12-09 12:02:59,920::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds.
>VM Channels Listener::INFO::2012-12-09 12:02:59,930::vmChannels::127::vds::(run) Starting VM channels listener thread.
>KsmMonitor::DEBUG::2012-12-09 12:02:59,943::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>KsmMonitor::DEBUG::2012-12-09 12:02:59,943::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/service ksm start' (cwd None)
>KsmMonitor::DEBUG::2012-12-09 12:02:59,962::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-12::DEBUG::2012-12-09 12:03:00,002::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '  No volume groups found\n'; <rc> = 0
>Thread-12::DEBUG::2012-12-09 12:03:00,002::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-12::DEBUG::2012-12-09 12:03:00,003::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-12::DEBUG::2012-12-09 12:03:00,003::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-12::DEBUG::2012-12-09 12:03:00,003::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-12::DEBUG::2012-12-09 12:03:00,003::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-12::DEBUG::2012-12-09 12:03:00,035::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-12::DEBUG::2012-12-09 12:03:00,035::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-12::DEBUG::2012-12-09 12:03:00,036::supervdsm::100::SuperVdsmProxy::(_start) Launching Super Vdsm
>Thread-12::DEBUG::2012-12-09 12:03:00,036::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py a4ee6697-3b3f-4000-88cc-f73bc31a7f74 2340' (cwd None)
>MainThread::DEBUG::2012-12-09 12:03:00,198::supervdsmServer::271::SuperVdsm.Server::(main) Making sure I'm root
>MainThread::DEBUG::2012-12-09 12:03:00,198::supervdsmServer::275::SuperVdsm.Server::(main) Parsing cmd args
>MainThread::DEBUG::2012-12-09 12:03:00,198::supervdsmServer::278::SuperVdsm.Server::(main) Creating PID file
>MainThread::DEBUG::2012-12-09 12:03:00,199::supervdsmServer::285::SuperVdsm.Server::(main) Cleaning old socket
>MainThread::DEBUG::2012-12-09 12:03:00,199::supervdsmServer::289::SuperVdsm.Server::(main) Setting up keep alive thread
>MainThread::DEBUG::2012-12-09 12:03:00,199::supervdsmServer::294::SuperVdsm.Server::(main) Creating remote object manager
>MainThread::DEBUG::2012-12-09 12:03:00,201::supervdsmServer::306::SuperVdsm.Server::(main) Started serving super vdsm object
>Thread-12::DEBUG::2012-12-09 12:03:02,042::supervdsm::153::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm
>MainProcess|Thread-12::DEBUG::2012-12-09 12:03:02,090::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-12::DEBUG::2012-12-09 12:03:03,094::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-12::DEBUG::2012-12-09 12:03:04,097::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-12::DEBUG::2012-12-09 12:03:06,100::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-12::DEBUG::2012-12-09 12:03:06,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-12::DEBUG::2012-12-09 12:03:06,119::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,119::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,119::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,119::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,120::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,120::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-12::DEBUG::2012-12-09 12:03:06,120::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-12::WARNING::2012-12-09 12:03:06,120::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists
>Thread-14::DEBUG::2012-12-09 12:08:18,405::task::588::TaskManager.Task::(_updateState) Task=`4680741d-2a75-49e6-aad4-00b61444bf7e`::moving from state init -> state preparing
>Thread-14::INFO::2012-12-09 12:08:18,405::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-14::INFO::2012-12-09 12:08:18,405::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-14::DEBUG::2012-12-09 12:08:18,406::task::1172::TaskManager.Task::(prepare) Task=`4680741d-2a75-49e6-aad4-00b61444bf7e`::finished: {}
>Thread-14::DEBUG::2012-12-09 12:08:18,406::task::588::TaskManager.Task::(_updateState) Task=`4680741d-2a75-49e6-aad4-00b61444bf7e`::moving from state preparing -> state finished
>Thread-14::DEBUG::2012-12-09 12:08:18,406::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-14::DEBUG::2012-12-09 12:08:18,406::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-14::DEBUG::2012-12-09 12:08:18,406::task::978::TaskManager.Task::(_decref) Task=`4680741d-2a75-49e6-aad4-00b61444bf7e`::ref 0 aborting False
>Thread-15::DEBUG::2012-12-09 12:08:18,438::BindingXMLRPC::894::vds::(wrapper) client [172.31.249.200]::call getCapabilities with () {}
>Thread-15::DEBUG::2012-12-09 12:08:18,714::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15::DEBUG::2012-12-09 12:08:18,793::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-15::DEBUG::2012-12-09 12:08:18,794::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15::DEBUG::2012-12-09 12:08:18,875::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-15::DEBUG::2012-12-09 12:08:18,876::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15::DEBUG::2012-12-09 12:08:18,995::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-15::DEBUG::2012-12-09 12:08:18,995::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15::DEBUG::2012-12-09 12:08:19,033::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-15::DEBUG::2012-12-09 12:08:19,033::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-15::DEBUG::2012-12-09 12:08:19,058::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-15::DEBUG::2012-12-09 12:08:19,262::BindingXMLRPC::900::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:f253ba95dd8f'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.14.1.el6.x86_64', 'buildtime': 1350326691.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '44.0.el6_3', 'buildtime': '1353357597', 'version': '4.9.6'}, 'qemu-kvm': {'release': '2.295.el6_3.8', 'buildtime': '1353000551', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.6', 'buildtime': '1352736961', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.8', 'buildtime': '1353000551', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel Celeron_4x0 (Conroe/Merom Class Core 2)', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '172.31.249.2', 'cfg': {'IPV6INIT': 'no', 'LAST_CONNECT': '1352852681', 'DEFROUTE': 'yes', 'DNS1': '172.31.249.254', 'IPADDR': '172.31.249.2', 'PREFIX': '24', 'GATEWAY': '172.31.249.254', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'DEVICE': 'rhevm', 'IPV4_FAILURE_FATAL': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes', 'UUID': 'd4a62fa4-f39e-49ea-8446-cd6d223ad220'}, 'mtu': '1500', 'netmask': '255.255.255.0', 'stp': 'off', 'bridged': True, 'gateway': '172.31.249.254', 'ports': ['eth0']}, 'nfs': {'iface': 'nfs', 'addr': '192.168.2.2', 'cfg': {'IPADDR': '192.168.2.2', 'MTU': '9000', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'nfs', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '9000', 'netmask': '255.255.255.0', 'stp': 'off', 'bridged': True, 'gateway': '0.0.0.0', 'ports': ['eth3']}}, 'bridges': {'rhevm': {'addr': '172.31.249.2', 'cfg': {'IPV6INIT': 'no', 'LAST_CONNECT': '1352852681', 'DEFROUTE': 'yes', 'DNS1': '172.31.249.254', 'IPADDR': '172.31.249.2', 'PREFIX': '24', 'GATEWAY': '172.31.249.254', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'DEVICE': 'rhevm', 'IPV4_FAILURE_FATAL': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes', 'UUID': 'd4a62fa4-f39e-49ea-8446-cd6d223ad220'}, 'mtu': '1500', 'netmask': '255.255.255.0', 'stp': 'off', 'ports': ['eth0']}, 'nfs': {'addr': '192.168.2.2', 'cfg': {'IPADDR': '192.168.2.2', 'MTU': '9000', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'nfs', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '9000', 'netmask': '255.255.255.0', 'stp': 'off', 'ports': ['eth3']}}, 'uuid': '3AE6D083-DD0B-F7CB-E10E-B3B512431CD4_52:54:00:09:2C:6F', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'addr': '', 'cfg': {'BRIDGE': 'nfs', 'NM_CONTROLLED': 'no', 'MTU': '9000', 'HWADDR': '52:54:00:19:21:66', 'STP': 'no', 'DEVICE': 'eth3', 'ONBOOT': 'yes'}, 'mtu': '9000', 'netmask': '', 'hwaddr': '52:54:00:19:21:66', 'speed': 0}, 'eth2': {'addr': '192.168.1.2', 'cfg': {'UUID': 'b93fbd20-b098-41eb-936a-281dd4b3c321', 'NM_CONTROLLED': 'yes', 'IPADDR': '192.168.1.2', 'MTU': '9000', 'HWADDR': '52:54:00:09:2C:6F', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'DEVICE': 'eth2', 'TYPE': 'Ethernet', 'ONBOOT': 'yes'}, 'mtu': '9000', 'netmask': '255.255.255.0', 'hwaddr': '52:54:00:09:2C:6F', 'speed': 0}, 'eth1': {'addr': '192.168.0.2', 'cfg': {'UUID': 'c68f3a42-dc65-49e2-8353-c27a4c59ea26', 'NM_CONTROLLED': 'yes', 'IPADDR': '192.168.0.2', 'MTU': '9000', 'HWADDR': '52:54:00:19:F7:81', 'NETMASK': '255.255.255.0', 'BOOTPROTO': 'none', 'DEVICE': 'eth1', 'TYPE': 'Ethernet', 'ONBOOT': 'yes'}, 'mtu': '9000', 'netmask': '255.255.255.0', 'hwaddr': '52:54:00:19:F7:81', 'speed': 0}, 'eth0': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'IPV6INIT': 'no', 'NM_CONTROLLED': 'no', 'DNS1': '172.31.249.254', 'LAST_CONNECT': '1352852681', 'DEFROUTE': 'yes', 'IPV4_FAILURE_FATAL': 'yes', 'HWADDR': '52:54:00:AE:80:60', 'PREFIX': '24', 'DEVICE': 'eth0', 'ONBOOT': 'yes', 'UUID': 'd4a62fa4-f39e-49ea-8446-cd6d223ad220'}, 'mtu': '1500', 'netmask': '', 'hwaddr': '52:54:00:AE:80:60', 'speed': 0}}, 'software_revision': '44.0', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pse36,clflush,mmx,fxsr,sse,sse2,syscall,nx,lm,rep_good,unfair_spinlock,pni,vmx,ssse3,hypervisor,lahf_lm,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:f253ba95dd8f', 'netConfigDirty': 'False', 'memSize': '5851', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.9', 'cpuSpeed': '3400.022', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '2', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.3.0.3.el6', 'version': '6Server', 'name': 'RHEL'}, 'lastClient': '172.31.249.200'}}
>Thread-21::DEBUG::2012-12-09 12:08:30,076::task::588::TaskManager.Task::(_updateState) Task=`04562b79-d44c-456a-b236-c9a7278dc324`::moving from state init -> state preparing
>Thread-21::INFO::2012-12-09 12:08:30,076::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-21::INFO::2012-12-09 12:08:30,076::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-21::DEBUG::2012-12-09 12:08:30,077::task::1172::TaskManager.Task::(prepare) Task=`04562b79-d44c-456a-b236-c9a7278dc324`::finished: {}
>Thread-21::DEBUG::2012-12-09 12:08:30,077::task::588::TaskManager.Task::(_updateState) Task=`04562b79-d44c-456a-b236-c9a7278dc324`::moving from state preparing -> state finished
>Thread-21::DEBUG::2012-12-09 12:08:30,077::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-21::DEBUG::2012-12-09 12:08:30,077::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-21::DEBUG::2012-12-09 12:08:30,077::task::978::TaskManager.Task::(_decref) Task=`04562b79-d44c-456a-b236-c9a7278dc324`::ref 0 aborting False
>Thread-24::DEBUG::2012-12-09 12:08:32,323::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-24::DEBUG::2012-12-09 12:08:32,324::task::588::TaskManager.Task::(_updateState) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::moving from state init -> state preparing
>Thread-24::INFO::2012-12-09 12:08:32,324::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-24::ERROR::2012-12-09 12:08:32,325::task::853::TaskManager.Task::(_setError) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 573, in getSpmStatus
>    pool = self.getPool(spUUID)
>  File "/usr/share/vdsm/storage/hsm.py", line 301, in getPool
>    raise se.StoragePoolUnknown(spUUID)
>StoragePoolUnknown: Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)
>Thread-24::DEBUG::2012-12-09 12:08:32,350::task::872::TaskManager.Task::(_run) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::Task._run: e2eacd72-c516-4adb-87e7-851ad697890c ('4372535e-3f4d-11e2-8ba0-525400202852',) {} failed - stopping task
>Thread-24::DEBUG::2012-12-09 12:08:32,350::task::1199::TaskManager.Task::(stop) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::stopping in state preparing (force False)
>Thread-24::DEBUG::2012-12-09 12:08:32,351::task::978::TaskManager.Task::(_decref) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::ref 1 aborting True
>Thread-24::INFO::2012-12-09 12:08:32,351::task::1157::TaskManager.Task::(prepare) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309
>Thread-24::DEBUG::2012-12-09 12:08:32,352::task::1162::TaskManager.Task::(prepare) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::Prepare: aborted: Unknown pool id, pool not connected
>Thread-24::DEBUG::2012-12-09 12:08:32,352::task::978::TaskManager.Task::(_decref) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::ref 0 aborting True
>Thread-24::DEBUG::2012-12-09 12:08:32,352::task::913::TaskManager.Task::(_doAbort) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::Task._doAbort: force False
>Thread-24::DEBUG::2012-12-09 12:08:32,353::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-24::DEBUG::2012-12-09 12:08:32,353::task::588::TaskManager.Task::(_updateState) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::moving from state preparing -> state aborting
>Thread-24::DEBUG::2012-12-09 12:08:32,354::task::537::TaskManager.Task::(__state_aborting) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::_aborting: recover policy none
>Thread-24::DEBUG::2012-12-09 12:08:32,354::task::588::TaskManager.Task::(_updateState) Task=`e2eacd72-c516-4adb-87e7-851ad697890c`::moving from state aborting -> state failed
>Thread-24::DEBUG::2012-12-09 12:08:32,354::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-24::DEBUG::2012-12-09 12:08:32,355::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-24::ERROR::2012-12-09 12:08:32,355::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)", 'code': 309}}
>Thread-25::DEBUG::2012-12-09 12:08:32,429::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-25::DEBUG::2012-12-09 12:08:32,430::task::588::TaskManager.Task::(_updateState) Task=`42105a26-34cb-47df-b8c4-b659458de878`::moving from state init -> state preparing
>Thread-25::INFO::2012-12-09 12:08:32,430::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', hostID=2, scsiKey='4372535e-3f4d-11e2-8ba0-525400202852', msdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', masterVersion=1, options=None)
>Thread-25::DEBUG::2012-12-09 12:08:32,430::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`802ad67d-ce61-4332-86e4-7a525b8f8ebb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-25::DEBUG::2012-12-09 12:08:32,431::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-25::DEBUG::2012-12-09 12:08:32,431::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-25::DEBUG::2012-12-09 12:08:32,431::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`802ad67d-ce61-4332-86e4-7a525b8f8ebb`::Granted request
>Thread-25::INFO::2012-12-09 12:08:32,431::sp::632::Storage.StoragePool::(connect) Connect host #2 to the storage pool 4372535e-3f4d-11e2-8ba0-525400202852 with master domain: c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (ver = 1)
>Thread-25::DEBUG::2012-12-09 12:08:32,432::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-25::DEBUG::2012-12-09 12:08:32,432::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-25::DEBUG::2012-12-09 12:08:32,432::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-25::DEBUG::2012-12-09 12:08:32,432::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-25::DEBUG::2012-12-09 12:08:32,433::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-25::DEBUG::2012-12-09 12:08:32,446::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-25::DEBUG::2012-12-09 12:08:32,447::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-25::DEBUG::2012-12-09 12:08:32,450::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-25::DEBUG::2012-12-09 12:08:33,456::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-25::DEBUG::2012-12-09 12:08:34,460::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-25::DEBUG::2012-12-09 12:08:36,463::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-25::DEBUG::2012-12-09 12:08:36,488::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-25::DEBUG::2012-12-09 12:08:36,489::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,489::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,490::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,490::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,490::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,490::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,490::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-25::DEBUG::2012-12-09 12:08:36,491::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,491::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-25::DEBUG::2012-12-09 12:08:36,586::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c294c8b1-b2e7-4564-bbb2-2d6c1f54b895" not found\n'; <rc> = 5
>Thread-25::WARNING::2012-12-09 12:08:36,587::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c294c8b1-b2e7-4564-bbb2-2d6c1f54b895" not found']
>Thread-25::DEBUG::2012-12-09 12:08:36,587::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-25::DEBUG::2012-12-09 12:08:36,588::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-25::DEBUG::2012-12-09 12:08:36,588::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-25::DEBUG::2012-12-09 12:08:36,588::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-25::DEBUG::2012-12-09 12:08:36,589::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-25::ERROR::2012-12-09 12:08:36,589::task::853::TaskManager.Task::(_setError) Task=`42105a26-34cb-47df-b8c4-b659458de878`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 862, in connectStoragePool
>    return self._connectStoragePool(spUUID, hostID, scsiKey, msdUUID, masterVersion, options)
>  File "/usr/share/vdsm/storage/hsm.py", line 904, in _connectStoragePool
>    res = pool.connect(hostID, scsiKey, msdUUID, masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 648, in connect
>    self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 1178, in __rebuild
>    self.masterDomain = self.getMasterDomain(msdUUID=msdUUID, masterVersion=masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 1522, in getMasterDomain
>    raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)
>StoragePoolMasterNotFound: Cannot find master domain: 'spUUID=4372535e-3f4d-11e2-8ba0-525400202852, msdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'
>Thread-25::DEBUG::2012-12-09 12:08:36,605::task::872::TaskManager.Task::(_run) Task=`42105a26-34cb-47df-b8c4-b659458de878`::Task._run: 42105a26-34cb-47df-b8c4-b659458de878 ('4372535e-3f4d-11e2-8ba0-525400202852', 2, '4372535e-3f4d-11e2-8ba0-525400202852', 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 1) {} failed - stopping task
>Thread-25::DEBUG::2012-12-09 12:08:36,605::task::1199::TaskManager.Task::(stop) Task=`42105a26-34cb-47df-b8c4-b659458de878`::stopping in state preparing (force False)
>Thread-25::DEBUG::2012-12-09 12:08:36,605::task::978::TaskManager.Task::(_decref) Task=`42105a26-34cb-47df-b8c4-b659458de878`::ref 1 aborting True
>Thread-25::INFO::2012-12-09 12:08:36,606::task::1157::TaskManager.Task::(prepare) Task=`42105a26-34cb-47df-b8c4-b659458de878`::aborting: Task is aborted: 'Cannot find master domain' - code 304
>Thread-25::DEBUG::2012-12-09 12:08:36,606::task::1162::TaskManager.Task::(prepare) Task=`42105a26-34cb-47df-b8c4-b659458de878`::Prepare: aborted: Cannot find master domain
>Thread-25::DEBUG::2012-12-09 12:08:36,606::task::978::TaskManager.Task::(_decref) Task=`42105a26-34cb-47df-b8c4-b659458de878`::ref 0 aborting True
>Thread-25::DEBUG::2012-12-09 12:08:36,606::task::913::TaskManager.Task::(_doAbort) Task=`42105a26-34cb-47df-b8c4-b659458de878`::Task._doAbort: force False
>Thread-25::DEBUG::2012-12-09 12:08:36,606::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-25::DEBUG::2012-12-09 12:08:36,607::task::588::TaskManager.Task::(_updateState) Task=`42105a26-34cb-47df-b8c4-b659458de878`::moving from state preparing -> state aborting
>Thread-25::DEBUG::2012-12-09 12:08:36,607::task::537::TaskManager.Task::(__state_aborting) Task=`42105a26-34cb-47df-b8c4-b659458de878`::_aborting: recover policy none
>Thread-25::DEBUG::2012-12-09 12:08:36,607::task::588::TaskManager.Task::(_updateState) Task=`42105a26-34cb-47df-b8c4-b659458de878`::moving from state aborting -> state failed
>Thread-25::DEBUG::2012-12-09 12:08:36,607::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-25::DEBUG::2012-12-09 12:08:36,607::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-25::ERROR::2012-12-09 12:08:36,607::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot find master domain: 'spUUID=4372535e-3f4d-11e2-8ba0-525400202852, msdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'", 'code': 304}}
>Thread-28::DEBUG::2012-12-09 12:08:37,492::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-28::DEBUG::2012-12-09 12:08:37,493::task::588::TaskManager.Task::(_updateState) Task=`5690feb5-f150-4f6d-bc5b-b4299cacea9c`::moving from state init -> state preparing
>Thread-28::INFO::2012-12-09 12:08:37,493::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-28::DEBUG::2012-12-09 12:08:37,493::taskManager::103::TaskManager::(getAllTasksStatuses) Entry.
>Thread-28::DEBUG::2012-12-09 12:08:37,493::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-28::INFO::2012-12-09 12:08:37,494::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}}
>Thread-28::DEBUG::2012-12-09 12:08:37,494::task::1172::TaskManager.Task::(prepare) Task=`5690feb5-f150-4f6d-bc5b-b4299cacea9c`::finished: {'allTasksStatus': {}}
>Thread-28::DEBUG::2012-12-09 12:08:37,495::task::588::TaskManager.Task::(_updateState) Task=`5690feb5-f150-4f6d-bc5b-b4299cacea9c`::moving from state preparing -> state finished
>Thread-28::DEBUG::2012-12-09 12:08:37,495::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-28::DEBUG::2012-12-09 12:08:37,495::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-28::DEBUG::2012-12-09 12:08:37,496::task::978::TaskManager.Task::(_decref) Task=`5690feb5-f150-4f6d-bc5b-b4299cacea9c`::ref 0 aborting False
>Thread-29::DEBUG::2012-12-09 12:08:37,522::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-29::DEBUG::2012-12-09 12:08:37,523::task::588::TaskManager.Task::(_updateState) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::moving from state init -> state preparing
>Thread-29::INFO::2012-12-09 12:08:37,523::logUtils::37::dispatcher::(wrapper) Run and protect: spmStop(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-29::DEBUG::2012-12-09 12:08:37,524::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4aead43-c602-4c93-91b8-c10bc4ed67af`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-29::DEBUG::2012-12-09 12:08:37,524::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-29::DEBUG::2012-12-09 12:08:37,525::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-29::DEBUG::2012-12-09 12:08:37,525::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4aead43-c602-4c93-91b8-c10bc4ed67af`::Granted request
>Thread-29::DEBUG::2012-12-09 12:08:37,526::task::817::TaskManager.Task::(resourceAcquired) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (exclusive)
>Thread-29::DEBUG::2012-12-09 12:08:37,526::task::978::TaskManager.Task::(_decref) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::ref 1 aborting False
>Thread-29::ERROR::2012-12-09 12:08:37,527::task::853::TaskManager.Task::(_setError) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 568, in spmStop
>    pool = self.getPool(spUUID)
>  File "/usr/share/vdsm/storage/hsm.py", line 301, in getPool
>    raise se.StoragePoolUnknown(spUUID)
>StoragePoolUnknown: Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)
>Thread-29::DEBUG::2012-12-09 12:08:37,527::task::872::TaskManager.Task::(_run) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::Task._run: 091753bf-f7dd-4c7c-b3bc-15854c74a784 ('4372535e-3f4d-11e2-8ba0-525400202852',) {} failed - stopping task
>Thread-29::DEBUG::2012-12-09 12:08:37,528::task::1199::TaskManager.Task::(stop) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::stopping in state preparing (force False)
>Thread-29::DEBUG::2012-12-09 12:08:37,528::task::978::TaskManager.Task::(_decref) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::ref 1 aborting True
>Thread-29::INFO::2012-12-09 12:08:37,529::task::1157::TaskManager.Task::(prepare) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309
>Thread-29::DEBUG::2012-12-09 12:08:37,529::task::1162::TaskManager.Task::(prepare) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::Prepare: aborted: Unknown pool id, pool not connected
>Thread-29::DEBUG::2012-12-09 12:08:37,529::task::978::TaskManager.Task::(_decref) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::ref 0 aborting True
>Thread-29::DEBUG::2012-12-09 12:08:37,530::task::913::TaskManager.Task::(_doAbort) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::Task._doAbort: force False
>Thread-29::DEBUG::2012-12-09 12:08:37,530::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-29::DEBUG::2012-12-09 12:08:37,530::task::588::TaskManager.Task::(_updateState) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::moving from state preparing -> state aborting
>Thread-29::DEBUG::2012-12-09 12:08:37,530::task::537::TaskManager.Task::(__state_aborting) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::_aborting: recover policy none
>Thread-29::DEBUG::2012-12-09 12:08:37,530::task::588::TaskManager.Task::(_updateState) Task=`091753bf-f7dd-4c7c-b3bc-15854c74a784`::moving from state aborting -> state failed
>Thread-29::DEBUG::2012-12-09 12:08:37,530::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-29::DEBUG::2012-12-09 12:08:37,531::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-29::DEBUG::2012-12-09 12:08:37,531::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-29::DEBUG::2012-12-09 12:08:37,531::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-29::DEBUG::2012-12-09 12:08:37,531::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-29::DEBUG::2012-12-09 12:08:37,531::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-29::ERROR::2012-12-09 12:08:37,531::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)", 'code': 309}}
>Thread-31::DEBUG::2012-12-09 12:08:40,460::task::588::TaskManager.Task::(_updateState) Task=`01fd6c24-1d37-498c-b11b-87faf5dcd9ad`::moving from state init -> state preparing
>Thread-31::INFO::2012-12-09 12:08:40,460::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-31::INFO::2012-12-09 12:08:40,460::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-31::DEBUG::2012-12-09 12:08:40,460::task::1172::TaskManager.Task::(prepare) Task=`01fd6c24-1d37-498c-b11b-87faf5dcd9ad`::finished: {}
>Thread-31::DEBUG::2012-12-09 12:08:40,461::task::588::TaskManager.Task::(_updateState) Task=`01fd6c24-1d37-498c-b11b-87faf5dcd9ad`::moving from state preparing -> state finished
>Thread-31::DEBUG::2012-12-09 12:08:40,461::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-31::DEBUG::2012-12-09 12:08:40,461::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-31::DEBUG::2012-12-09 12:08:40,461::task::978::TaskManager.Task::(_decref) Task=`01fd6c24-1d37-498c-b11b-87faf5dcd9ad`::ref 0 aborting False
>Thread-37::DEBUG::2012-12-09 12:08:50,727::task::588::TaskManager.Task::(_updateState) Task=`cc4fd281-c9aa-4747-a202-35cb73442377`::moving from state init -> state preparing
>Thread-37::INFO::2012-12-09 12:08:50,728::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-37::INFO::2012-12-09 12:08:50,728::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-37::DEBUG::2012-12-09 12:08:50,728::task::1172::TaskManager.Task::(prepare) Task=`cc4fd281-c9aa-4747-a202-35cb73442377`::finished: {}
>Thread-37::DEBUG::2012-12-09 12:08:50,728::task::588::TaskManager.Task::(_updateState) Task=`cc4fd281-c9aa-4747-a202-35cb73442377`::moving from state preparing -> state finished
>Thread-37::DEBUG::2012-12-09 12:08:50,728::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-37::DEBUG::2012-12-09 12:08:50,729::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-37::DEBUG::2012-12-09 12:08:50,729::task::978::TaskManager.Task::(_decref) Task=`cc4fd281-c9aa-4747-a202-35cb73442377`::ref 0 aborting False
>Thread-39::DEBUG::2012-12-09 12:08:51,157::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-39::DEBUG::2012-12-09 12:08:51,158::task::588::TaskManager.Task::(_updateState) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::moving from state init -> state preparing
>Thread-39::INFO::2012-12-09 12:08:51,158::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-39::ERROR::2012-12-09 12:08:51,158::task::853::TaskManager.Task::(_setError) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 573, in getSpmStatus
>    pool = self.getPool(spUUID)
>  File "/usr/share/vdsm/storage/hsm.py", line 301, in getPool
>    raise se.StoragePoolUnknown(spUUID)
>StoragePoolUnknown: Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)
>Thread-39::DEBUG::2012-12-09 12:08:51,159::task::872::TaskManager.Task::(_run) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::Task._run: ffca7606-970c-44c8-942b-4f1913146d47 ('4372535e-3f4d-11e2-8ba0-525400202852',) {} failed - stopping task
>Thread-39::DEBUG::2012-12-09 12:08:51,159::task::1199::TaskManager.Task::(stop) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::stopping in state preparing (force False)
>Thread-39::DEBUG::2012-12-09 12:08:51,159::task::978::TaskManager.Task::(_decref) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::ref 1 aborting True
>Thread-39::INFO::2012-12-09 12:08:51,159::task::1157::TaskManager.Task::(prepare) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309
>Thread-39::DEBUG::2012-12-09 12:08:51,159::task::1162::TaskManager.Task::(prepare) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::Prepare: aborted: Unknown pool id, pool not connected
>Thread-39::DEBUG::2012-12-09 12:08:51,160::task::978::TaskManager.Task::(_decref) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::ref 0 aborting True
>Thread-39::DEBUG::2012-12-09 12:08:51,160::task::913::TaskManager.Task::(_doAbort) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::Task._doAbort: force False
>Thread-39::DEBUG::2012-12-09 12:08:51,160::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-39::DEBUG::2012-12-09 12:08:51,160::task::588::TaskManager.Task::(_updateState) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::moving from state preparing -> state aborting
>Thread-39::DEBUG::2012-12-09 12:08:51,160::task::537::TaskManager.Task::(__state_aborting) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::_aborting: recover policy none
>Thread-39::DEBUG::2012-12-09 12:08:51,160::task::588::TaskManager.Task::(_updateState) Task=`ffca7606-970c-44c8-942b-4f1913146d47`::moving from state aborting -> state failed
>Thread-39::DEBUG::2012-12-09 12:08:51,161::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-39::DEBUG::2012-12-09 12:08:51,161::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-39::ERROR::2012-12-09 12:08:51,161::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)", 'code': 309}}
>Thread-40::DEBUG::2012-12-09 12:08:51,228::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-40::DEBUG::2012-12-09 12:08:51,229::task::588::TaskManager.Task::(_updateState) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::moving from state init -> state preparing
>Thread-40::INFO::2012-12-09 12:08:51,229::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', hostID=2, scsiKey='4372535e-3f4d-11e2-8ba0-525400202852', msdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', masterVersion=1, options=None)
>Thread-40::DEBUG::2012-12-09 12:08:51,230::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`08445f71-8e35-485f-8c2d-ae05576aecf7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-40::DEBUG::2012-12-09 12:08:51,230::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-40::DEBUG::2012-12-09 12:08:51,230::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-40::DEBUG::2012-12-09 12:08:51,230::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`08445f71-8e35-485f-8c2d-ae05576aecf7`::Granted request
>Thread-40::INFO::2012-12-09 12:08:51,231::sp::632::Storage.StoragePool::(connect) Connect host #2 to the storage pool 4372535e-3f4d-11e2-8ba0-525400202852 with master domain: c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (ver = 1)
>Thread-40::DEBUG::2012-12-09 12:08:51,231::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-40::DEBUG::2012-12-09 12:08:51,231::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-40::DEBUG::2012-12-09 12:08:51,231::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-40::DEBUG::2012-12-09 12:08:51,231::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-40::DEBUG::2012-12-09 12:08:51,232::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-40::DEBUG::2012-12-09 12:08:51,243::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-40::DEBUG::2012-12-09 12:08:51,243::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-40::DEBUG::2012-12-09 12:08:51,245::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-40::DEBUG::2012-12-09 12:08:52,250::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-40::DEBUG::2012-12-09 12:08:52,253::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-40::DEBUG::2012-12-09 12:08:54,256::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-40::DEBUG::2012-12-09 12:08:54,283::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-40::DEBUG::2012-12-09 12:08:54,284::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,285::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,285::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,286::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,286::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,286::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,287::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-40::DEBUG::2012-12-09 12:08:54,287::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,289::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-40::DEBUG::2012-12-09 12:08:54,377::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c294c8b1-b2e7-4564-bbb2-2d6c1f54b895" not found\n'; <rc> = 5
>Thread-40::WARNING::2012-12-09 12:08:54,378::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c294c8b1-b2e7-4564-bbb2-2d6c1f54b895" not found']
>Thread-40::DEBUG::2012-12-09 12:08:54,378::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-40::DEBUG::2012-12-09 12:08:54,379::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-40::DEBUG::2012-12-09 12:08:54,379::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-40::DEBUG::2012-12-09 12:08:54,379::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-40::DEBUG::2012-12-09 12:08:54,379::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-40::ERROR::2012-12-09 12:08:54,380::task::853::TaskManager.Task::(_setError) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 862, in connectStoragePool
>    return self._connectStoragePool(spUUID, hostID, scsiKey, msdUUID, masterVersion, options)
>  File "/usr/share/vdsm/storage/hsm.py", line 904, in _connectStoragePool
>    res = pool.connect(hostID, scsiKey, msdUUID, masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 648, in connect
>    self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 1178, in __rebuild
>    self.masterDomain = self.getMasterDomain(msdUUID=msdUUID, masterVersion=masterVersion)
>  File "/usr/share/vdsm/storage/sp.py", line 1522, in getMasterDomain
>    raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)
>StoragePoolMasterNotFound: Cannot find master domain: 'spUUID=4372535e-3f4d-11e2-8ba0-525400202852, msdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'
>Thread-40::DEBUG::2012-12-09 12:08:54,380::task::872::TaskManager.Task::(_run) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::Task._run: d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e ('4372535e-3f4d-11e2-8ba0-525400202852', 2, '4372535e-3f4d-11e2-8ba0-525400202852', 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 1) {} failed - stopping task
>Thread-40::DEBUG::2012-12-09 12:08:54,380::task::1199::TaskManager.Task::(stop) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::stopping in state preparing (force False)
>Thread-40::DEBUG::2012-12-09 12:08:54,380::task::978::TaskManager.Task::(_decref) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::ref 1 aborting True
>Thread-40::INFO::2012-12-09 12:08:54,381::task::1157::TaskManager.Task::(prepare) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::aborting: Task is aborted: 'Cannot find master domain' - code 304
>Thread-40::DEBUG::2012-12-09 12:08:54,381::task::1162::TaskManager.Task::(prepare) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::Prepare: aborted: Cannot find master domain
>Thread-40::DEBUG::2012-12-09 12:08:54,381::task::978::TaskManager.Task::(_decref) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::ref 0 aborting True
>Thread-40::DEBUG::2012-12-09 12:08:54,381::task::913::TaskManager.Task::(_doAbort) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::Task._doAbort: force False
>Thread-40::DEBUG::2012-12-09 12:08:54,381::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-40::DEBUG::2012-12-09 12:08:54,381::task::588::TaskManager.Task::(_updateState) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::moving from state preparing -> state aborting
>Thread-40::DEBUG::2012-12-09 12:08:54,382::task::537::TaskManager.Task::(__state_aborting) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::_aborting: recover policy none
>Thread-40::DEBUG::2012-12-09 12:08:54,382::task::588::TaskManager.Task::(_updateState) Task=`d2f80cdf-cb76-4cfc-ac97-2e514c9f6a1e`::moving from state aborting -> state failed
>Thread-40::DEBUG::2012-12-09 12:08:54,382::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-40::DEBUG::2012-12-09 12:08:54,382::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-40::ERROR::2012-12-09 12:08:54,382::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot find master domain: 'spUUID=4372535e-3f4d-11e2-8ba0-525400202852, msdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'", 'code': 304}}
>Thread-44::DEBUG::2012-12-09 12:08:58,091::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-44::DEBUG::2012-12-09 12:08:58,092::task::588::TaskManager.Task::(_updateState) Task=`db2acfd7-1fe1-4573-8257-affad03ea23d`::moving from state init -> state preparing
>Thread-44::INFO::2012-12-09 12:08:58,093::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='4372535e-3f4d-11e2-8ba0-525400202852', conList=[{'connection': '192.168.1.254', 'iqn': 'iqn.2012-11.net.selfip.icp:ian.1', 'portal': '1', 'user': '', 'password': '******', 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf', 'port': '3260'}, {'connection': '192.168.0.254', 'iqn': 'iqn.2012-11.net.selfip.icp:ian.1', 'portal': '1', 'user': '', 'password': '******', 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3', 'port': '3260'}], options=None)
>Thread-44::DEBUG::2012-12-09 12:08:58,094::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.1.254:3260 --op=new' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:58,117::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::DEBUG::2012-12-09 12:08:58,118::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.1.254:3260 -l' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:58,650::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::DEBUG::2012-12-09 12:08:58,651::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.1.254:3260 -n node.startup -v manual --op=update' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:58,667::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::DEBUG::2012-12-09 12:08:58,668::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.0.254:3260 --op=new' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:58,679::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::DEBUG::2012-12-09 12:08:58,679::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.0.254:3260 -l' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:59,205::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::DEBUG::2012-12-09 12:08:59,206::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-11.net.selfip.icp:ian.1 -I default -p 192.168.0.254:3260 -n node.startup -v manual --op=update' (cwd None)
>Thread-44::DEBUG::2012-12-09 12:08:59,308::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-44::INFO::2012-12-09 12:08:59,309::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf'}, {'status': 0, 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3'}]}
>Thread-44::DEBUG::2012-12-09 12:08:59,314::task::1172::TaskManager.Task::(prepare) Task=`db2acfd7-1fe1-4573-8257-affad03ea23d`::finished: {'statuslist': [{'status': 0, 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf'}, {'status': 0, 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3'}]}
>Thread-44::DEBUG::2012-12-09 12:08:59,314::task::588::TaskManager.Task::(_updateState) Task=`db2acfd7-1fe1-4573-8257-affad03ea23d`::moving from state preparing -> state finished
>Thread-44::DEBUG::2012-12-09 12:08:59,315::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-44::DEBUG::2012-12-09 12:08:59,315::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-44::DEBUG::2012-12-09 12:08:59,315::task::978::TaskManager.Task::(_decref) Task=`db2acfd7-1fe1-4573-8257-affad03ea23d`::ref 0 aborting False
>Thread-46::DEBUG::2012-12-09 12:08:59,408::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-46::DEBUG::2012-12-09 12:08:59,409::task::588::TaskManager.Task::(_updateState) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::moving from state init -> state preparing
>Thread-46::INFO::2012-12-09 12:08:59,409::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-46::ERROR::2012-12-09 12:08:59,409::task::853::TaskManager.Task::(_setError) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 573, in getSpmStatus
>    pool = self.getPool(spUUID)
>  File "/usr/share/vdsm/storage/hsm.py", line 301, in getPool
>    raise se.StoragePoolUnknown(spUUID)
>StoragePoolUnknown: Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)
>Thread-46::DEBUG::2012-12-09 12:08:59,410::task::872::TaskManager.Task::(_run) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::Task._run: ac94c7f2-ff1f-49ee-83c2-f838813e03f2 ('4372535e-3f4d-11e2-8ba0-525400202852',) {} failed - stopping task
>Thread-46::DEBUG::2012-12-09 12:08:59,410::task::1199::TaskManager.Task::(stop) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::stopping in state preparing (force False)
>Thread-46::DEBUG::2012-12-09 12:08:59,410::task::978::TaskManager.Task::(_decref) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::ref 1 aborting True
>Thread-46::INFO::2012-12-09 12:08:59,410::task::1157::TaskManager.Task::(prepare) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309
>Thread-46::DEBUG::2012-12-09 12:08:59,410::task::1162::TaskManager.Task::(prepare) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::Prepare: aborted: Unknown pool id, pool not connected
>Thread-46::DEBUG::2012-12-09 12:08:59,411::task::978::TaskManager.Task::(_decref) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::ref 0 aborting True
>Thread-46::DEBUG::2012-12-09 12:08:59,411::task::913::TaskManager.Task::(_doAbort) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::Task._doAbort: force False
>Thread-46::DEBUG::2012-12-09 12:08:59,411::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-46::DEBUG::2012-12-09 12:08:59,411::task::588::TaskManager.Task::(_updateState) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::moving from state preparing -> state aborting
>Thread-46::DEBUG::2012-12-09 12:08:59,411::task::537::TaskManager.Task::(__state_aborting) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::_aborting: recover policy none
>Thread-46::DEBUG::2012-12-09 12:08:59,412::task::588::TaskManager.Task::(_updateState) Task=`ac94c7f2-ff1f-49ee-83c2-f838813e03f2`::moving from state aborting -> state failed
>Thread-46::DEBUG::2012-12-09 12:08:59,412::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-46::DEBUG::2012-12-09 12:08:59,412::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-46::ERROR::2012-12-09 12:08:59,412::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('4372535e-3f4d-11e2-8ba0-525400202852',)", 'code': 309}}
>Thread-47::DEBUG::2012-12-09 12:08:59,532::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-47::DEBUG::2012-12-09 12:08:59,533::task::588::TaskManager.Task::(_updateState) Task=`e4b1875e-f595-43f7-a9b7-b1c1761b4dff`::moving from state init -> state preparing
>Thread-47::INFO::2012-12-09 12:08:59,533::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', hostID=2, scsiKey='4372535e-3f4d-11e2-8ba0-525400202852', msdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', masterVersion=1, options=None)
>Thread-47::DEBUG::2012-12-09 12:08:59,534::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25245e15-087c-4c0e-83a9-3c594fe5c15b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-47::DEBUG::2012-12-09 12:08:59,534::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-47::DEBUG::2012-12-09 12:08:59,534::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-47::DEBUG::2012-12-09 12:08:59,535::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25245e15-087c-4c0e-83a9-3c594fe5c15b`::Granted request
>Thread-47::INFO::2012-12-09 12:08:59,535::sp::632::Storage.StoragePool::(connect) Connect host #2 to the storage pool 4372535e-3f4d-11e2-8ba0-525400202852 with master domain: c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (ver = 1)
>Thread-47::DEBUG::2012-12-09 12:08:59,535::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-47::DEBUG::2012-12-09 12:08:59,536::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-47::DEBUG::2012-12-09 12:08:59,536::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-47::DEBUG::2012-12-09 12:08:59,536::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-47::DEBUG::2012-12-09 12:08:59,536::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:08:59,551::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:08:59,552::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-47::DEBUG::2012-12-09 12:08:59,557::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-47::DEBUG::2012-12-09 12:08:59,561::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-47::DEBUG::2012-12-09 12:08:59,564::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host2/scan' (cwd None)
>MainProcess|Thread-47::DEBUG::2012-12-09 12:08:59,567::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host3/scan' (cwd None)
>MainProcess|Thread-47::DEBUG::2012-12-09 12:08:59,572::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-48::DEBUG::2012-12-09 12:09:01,004::task::588::TaskManager.Task::(_updateState) Task=`5dcc711e-c6a8-4ecf-aa42-b96bc6e608ab`::moving from state init -> state preparing
>Thread-48::INFO::2012-12-09 12:09:01,005::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-48::INFO::2012-12-09 12:09:01,006::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
>Thread-48::DEBUG::2012-12-09 12:09:01,006::task::1172::TaskManager.Task::(prepare) Task=`5dcc711e-c6a8-4ecf-aa42-b96bc6e608ab`::finished: {}
>Thread-48::DEBUG::2012-12-09 12:09:01,006::task::588::TaskManager.Task::(_updateState) Task=`5dcc711e-c6a8-4ecf-aa42-b96bc6e608ab`::moving from state preparing -> state finished
>Thread-48::DEBUG::2012-12-09 12:09:01,007::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-48::DEBUG::2012-12-09 12:09:01,007::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-48::DEBUG::2012-12-09 12:09:01,007::task::978::TaskManager.Task::(_decref) Task=`5dcc711e-c6a8-4ecf-aa42-b96bc6e608ab`::ref 0 aborting False
>Thread-47::DEBUG::2012-12-09 12:09:01,575::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:01,609::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:01,611::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,611::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,611::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,611::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,611::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,612::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,612::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-47::DEBUG::2012-12-09 12:09:01,612::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,613::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:01,772::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:01,773::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:01,773::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-47::DEBUG::2012-12-09 12:09:01,774::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,009::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,009::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,010::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,010::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,020::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000716322 s, 2.9 MB/s\n'; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,020::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000716322 s, 2.9 MB/s'], size: 2048
>Thread-47::DEBUG::2012-12-09 12:09:02,020::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-47::WARNING::2012-12-09 12:09:02,021::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-47::DEBUG::2012-12-09 12:09:02,021::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-47::DEBUG::2012-12-09 12:09:02,021::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,021::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,022::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,022::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,023::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,124::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,125::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,125::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>Thread-47::DEBUG::2012-12-09 12:09:02,126::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/ids' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,322::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,322::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,322::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/leases' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,524::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,524::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,525::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,525::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/inbox c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/outbox c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,738::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,739::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,739::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,740::lvm::319::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,740::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,847::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,847::lvm::342::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,848::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS'
>Thread-47::DEBUG::2012-12-09 12:09:02,848::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS'
>Thread-47::DEBUG::2012-12-09 12:09:02,848::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS'
>Thread-47::DEBUG::2012-12-09 12:09:02,849::sp::1539::Storage.StoragePool::(getMasterDomain) Master domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 verified, version 1
>Thread-47::DEBUG::2012-12-09 12:09:02,849::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>Thread-47::DEBUG::2012-12-09 12:09:02,849::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-47::DEBUG::2012-12-09 12:09:02,850::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,850::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,850::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,850::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,850::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,851::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,955::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:02,957::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,957::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>Thread-50::DEBUG::2012-12-09 12:09:02,959::domainMonitor::143::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-47::DEBUG::2012-12-09 12:09:02,959::sp::1573::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `4372535e-3f4d-11e2-8ba0-525400202852` started monitoring domain `c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`
>Thread-50::DEBUG::2012-12-09 12:09:02,960::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:02,966::domainMonitor::143::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-47::DEBUG::2012-12-09 12:09:02,966::sp::1573::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `4372535e-3f4d-11e2-8ba0-525400202852` started monitoring domain `c080b7a1-1d5f-449f-9a65-375961d37141`
>Thread-51::DEBUG::2012-12-09 12:09:02,968::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,968::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-51::DEBUG::2012-12-09 12:09:02,969::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:02,976::lvm::352::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-47::DEBUG::2012-12-09 12:09:02,976::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:03,076::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:03,078::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::ERROR::2012-12-09 12:09:03,078::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-51::DEBUG::2012-12-09 12:09:03,092::domainMonitor::216::Storage.DomainMonitorThread::(_monitorDomain) Domain c080b7a1-1d5f-449f-9a65-375961d37141 changed its status to Invalid
>Thread-50::DEBUG::2012-12-09 12:09:03,111::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:09:03,112::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:03,125::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-47::WARNING::2012-12-09 12:09:03,127::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-47::DEBUG::2012-12-09 12:09:03,128::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:09:03,133::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000503256 s, 8.1 MB/s\n'; <rc> = 0
>Thread-47::INFO::2012-12-09 12:09:03,134::sp::1133::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 to /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-47::INFO::2012-12-09 12:09:03,136::sp::1133::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 to /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd
>Thread-47::DEBUG::2012-12-09 12:09:03,138::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>Thread-50::INFO::2012-12-09 12:09:03,138::safelease::160::SANLock::(acquireHostId) Acquiring host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (id: 2)
>Thread-50::DEBUG::2012-12-09 12:09:03,141::safelease::178::SANLock::(acquireHostId) Host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 successfully acquired (id: 2)
>Thread-47::DEBUG::2012-12-09 12:09:03,143::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=16' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:03,152::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00255987 s, 1.6 MB/s\n'; <rc> = 0
>Thread-47::INFO::2012-12-09 12:09:03,153::storage_mailbox::340::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16']
>Thread-47::DEBUG::2012-12-09 12:09:03,153::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16' (cwd None)
>Thread-47::DEBUG::2012-12-09 12:09:03,164::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00227041 s, 1.8 MB/s\n'; <rc> = 0
>Thread-47::DEBUG::2012-12-09 12:09:03,165::storage_mailbox::181::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 4372535e-3f4d-11e2-8ba0-525400202852
>Thread-47::DEBUG::2012-12-09 12:09:03,166::sp::439::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 4372535e-3f4d-11e2-8ba0-525400202852 on master domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-47::DEBUG::2012-12-09 12:09:03,166::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-47::DEBUG::2012-12-09 12:09:03,167::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-47::DEBUG::2012-12-09 12:09:03,167::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-47::DEBUG::2012-12-09 12:09:03,167::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-47::INFO::2012-12-09 12:09:03,168::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True
>Thread-47::DEBUG::2012-12-09 12:09:03,168::task::1172::TaskManager.Task::(prepare) Task=`e4b1875e-f595-43f7-a9b7-b1c1761b4dff`::finished: True
>Thread-47::DEBUG::2012-12-09 12:09:03,169::task::588::TaskManager.Task::(_updateState) Task=`e4b1875e-f595-43f7-a9b7-b1c1761b4dff`::moving from state preparing -> state finished
>Thread-47::DEBUG::2012-12-09 12:09:03,169::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-47::DEBUG::2012-12-09 12:09:03,169::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-47::DEBUG::2012-12-09 12:09:03,170::task::978::TaskManager.Task::(_decref) Task=`e4b1875e-f595-43f7-a9b7-b1c1761b4dff`::ref 0 aborting False
>Thread-59::DEBUG::2012-12-09 12:09:03,192::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-59::DEBUG::2012-12-09 12:09:03,193::task::588::TaskManager.Task::(_updateState) Task=`22b42121-ddbf-4154-9a23-8ab898406950`::moving from state init -> state preparing
>Thread-59::INFO::2012-12-09 12:09:03,194::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-59::DEBUG::2012-12-09 12:09:03,195::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,195::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,196::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,196::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,196::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,197::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-59::DEBUG::2012-12-09 12:09:03,301::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-59::DEBUG::2012-12-09 12:09:03,302::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-59::DEBUG::2012-12-09 12:09:03,303::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>Thread-59::INFO::2012-12-09 12:09:03,303::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 48}}
>Thread-59::DEBUG::2012-12-09 12:09:03,304::task::1172::TaskManager.Task::(prepare) Task=`22b42121-ddbf-4154-9a23-8ab898406950`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 48}}
>Thread-59::DEBUG::2012-12-09 12:09:03,304::task::588::TaskManager.Task::(_updateState) Task=`22b42121-ddbf-4154-9a23-8ab898406950`::moving from state preparing -> state finished
>Thread-59::DEBUG::2012-12-09 12:09:03,304::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-59::DEBUG::2012-12-09 12:09:03,304::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-59::DEBUG::2012-12-09 12:09:03,305::task::978::TaskManager.Task::(_decref) Task=`22b42121-ddbf-4154-9a23-8ab898406950`::ref 0 aborting False
>Thread-60::DEBUG::2012-12-09 12:09:03,319::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-60::DEBUG::2012-12-09 12:09:03,320::task::588::TaskManager.Task::(_updateState) Task=`840e4a06-b72d-43e4-9c89-0f9ad6656e00`::moving from state init -> state preparing
>Thread-60::INFO::2012-12-09 12:09:03,320::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-60::DEBUG::2012-12-09 12:09:03,321::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,321::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,321::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,322::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,322::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,322::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-60::DEBUG::2012-12-09 12:09:03,419::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-60::DEBUG::2012-12-09 12:09:03,420::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-60::DEBUG::2012-12-09 12:09:03,420::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>Thread-60::INFO::2012-12-09 12:09:03,421::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 48}}
>Thread-60::DEBUG::2012-12-09 12:09:03,421::task::1172::TaskManager.Task::(prepare) Task=`840e4a06-b72d-43e4-9c89-0f9ad6656e00`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'Free', 'spmLver': 48}}
>Thread-60::DEBUG::2012-12-09 12:09:03,421::task::588::TaskManager.Task::(_updateState) Task=`840e4a06-b72d-43e4-9c89-0f9ad6656e00`::moving from state preparing -> state finished
>Thread-60::DEBUG::2012-12-09 12:09:03,421::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-60::DEBUG::2012-12-09 12:09:03,422::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-60::DEBUG::2012-12-09 12:09:03,422::task::978::TaskManager.Task::(_decref) Task=`840e4a06-b72d-43e4-9c89-0f9ad6656e00`::ref 0 aborting False
>Thread-61::DEBUG::2012-12-09 12:09:03,541::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-61::DEBUG::2012-12-09 12:09:03,542::task::588::TaskManager.Task::(_updateState) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::moving from state init -> state preparing
>Thread-61::INFO::2012-12-09 12:09:03,542::logUtils::37::dispatcher::(wrapper) Run and protect: spmStart(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', prevID=2, prevLVER='48', recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion='3', options=None)
>Thread-61::DEBUG::2012-12-09 12:09:03,544::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4467b8c-7678-4152-beb2-e1e34a5abdf2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-61::DEBUG::2012-12-09 12:09:03,544::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-61::DEBUG::2012-12-09 12:09:03,545::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-61::DEBUG::2012-12-09 12:09:03,545::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4467b8c-7678-4152-beb2-e1e34a5abdf2`::Granted request
>Thread-61::DEBUG::2012-12-09 12:09:03,546::task::817::TaskManager.Task::(resourceAcquired) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (exclusive)
>Thread-61::DEBUG::2012-12-09 12:09:03,546::task::978::TaskManager.Task::(_decref) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::ref 1 aborting False
>Thread-61::INFO::2012-12-09 12:09:03,547::logUtils::39::dispatcher::(wrapper) Run and protect: spmStart, Return response: None
>Thread-61::DEBUG::2012-12-09 12:09:03,547::task::1167::TaskManager.Task::(prepare) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::Prepare: 1 jobs exist, move to acquiring
>Thread-61::DEBUG::2012-12-09 12:09:03,547::task::588::TaskManager.Task::(_updateState) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::moving from state preparing -> state acquiring
>Thread-61::DEBUG::2012-12-09 12:09:03,548::task::588::TaskManager.Task::(_updateState) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::moving from state acquiring -> state queued
>Thread-61::DEBUG::2012-12-09 12:09:03,548::taskManager::48::TaskManager::(_queueTask) queueing task: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-6::DEBUG::2012-12-09 12:09:03,549::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,549::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 87f482a9-1b1a-4fe7-aa72-9cddf7717383 running: <bound method Task.commit of <storage.task.Task instance at 0x265e7a0>> with: None
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,549::task::1181::TaskManager.Task::(commit) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::Commiting task: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::task::588::TaskManager.Task::(_updateState) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::moving from state queued -> state running
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::task::892::TaskManager.Task::(_runJobs) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f3510417dd0>> (args: (2, '48', 'false', 250, 3) kwargs: {})
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::task::317::TaskManager.Task::(run) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f3510417dd0>> (args: (2, '48', 'false', 250, 3) kwargs: {}) callback None
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,550::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,551::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,551::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,551::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,551::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,552::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-61::DEBUG::2012-12-09 12:09:03,556::taskManager::54::TaskManager::(_queueTask) task queued: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-61::DEBUG::2012-12-09 12:09:03,556::task::1169::TaskManager.Task::(prepare) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::returning
>Thread-61::DEBUG::2012-12-09 12:09:03,556::task::978::TaskManager.Task::(_decref) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::ref 1 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,658::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,660::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,660::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,660::misc::1054::SamplingMethod::(__call__) Returning last result
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,661::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,661::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,661::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,661::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,662::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,662::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,770::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,770::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,770::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,771::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,771::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,771::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,771::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,772::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,883::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,885::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:09:03,886::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::INFO::2012-12-09 12:09:03,887::safelease::160::SANLock::(acquireHostId) Acquiring host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (id: 2)
>Thread-62::DEBUG::2012-12-09 12:09:04,587::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-62::DEBUG::2012-12-09 12:09:04,588::task::588::TaskManager.Task::(_updateState) Task=`c9c0a3e0-be9d-434c-bb2e-7cb0cecb5742`::moving from state init -> state preparing
>Thread-62::INFO::2012-12-09 12:09:04,588::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-62::DEBUG::2012-12-09 12:09:04,589::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-62::DEBUG::2012-12-09 12:09:04,589::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-62::INFO::2012-12-09 12:09:04,590::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-62::DEBUG::2012-12-09 12:09:04,590::task::1172::TaskManager.Task::(prepare) Task=`c9c0a3e0-be9d-434c-bb2e-7cb0cecb5742`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-62::DEBUG::2012-12-09 12:09:04,590::task::588::TaskManager.Task::(_updateState) Task=`c9c0a3e0-be9d-434c-bb2e-7cb0cecb5742`::moving from state preparing -> state finished
>Thread-62::DEBUG::2012-12-09 12:09:04,591::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-62::DEBUG::2012-12-09 12:09:04,591::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-62::DEBUG::2012-12-09 12:09:04,592::task::978::TaskManager.Task::(_decref) Task=`c9c0a3e0-be9d-434c-bb2e-7cb0cecb5742`::ref 0 aborting False
>Thread-64::DEBUG::2012-12-09 12:09:05,616::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-64::DEBUG::2012-12-09 12:09:05,617::task::588::TaskManager.Task::(_updateState) Task=`1d723496-64db-48ce-89bf-99dc769db177`::moving from state init -> state preparing
>Thread-64::INFO::2012-12-09 12:09:05,617::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-64::DEBUG::2012-12-09 12:09:05,617::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-64::DEBUG::2012-12-09 12:09:05,617::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-64::INFO::2012-12-09 12:09:05,617::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-64::DEBUG::2012-12-09 12:09:05,618::task::1172::TaskManager.Task::(prepare) Task=`1d723496-64db-48ce-89bf-99dc769db177`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-64::DEBUG::2012-12-09 12:09:05,618::task::588::TaskManager.Task::(_updateState) Task=`1d723496-64db-48ce-89bf-99dc769db177`::moving from state preparing -> state finished
>Thread-64::DEBUG::2012-12-09 12:09:05,618::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-64::DEBUG::2012-12-09 12:09:05,618::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-64::DEBUG::2012-12-09 12:09:05,618::task::978::TaskManager.Task::(_decref) Task=`1d723496-64db-48ce-89bf-99dc769db177`::ref 0 aborting False
>Thread-65::DEBUG::2012-12-09 12:09:06,644::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-65::DEBUG::2012-12-09 12:09:06,645::task::588::TaskManager.Task::(_updateState) Task=`b19ef029-819f-4b83-a7a2-38eb7f9d1a84`::moving from state init -> state preparing
>Thread-65::INFO::2012-12-09 12:09:06,646::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-65::DEBUG::2012-12-09 12:09:06,646::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-65::DEBUG::2012-12-09 12:09:06,646::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-65::INFO::2012-12-09 12:09:06,647::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-65::DEBUG::2012-12-09 12:09:06,647::task::1172::TaskManager.Task::(prepare) Task=`b19ef029-819f-4b83-a7a2-38eb7f9d1a84`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-65::DEBUG::2012-12-09 12:09:06,648::task::588::TaskManager.Task::(_updateState) Task=`b19ef029-819f-4b83-a7a2-38eb7f9d1a84`::moving from state preparing -> state finished
>Thread-65::DEBUG::2012-12-09 12:09:06,648::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-65::DEBUG::2012-12-09 12:09:06,648::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-65::DEBUG::2012-12-09 12:09:06,649::task::978::TaskManager.Task::(_decref) Task=`b19ef029-819f-4b83-a7a2-38eb7f9d1a84`::ref 0 aborting False
>Thread-67::DEBUG::2012-12-09 12:09:07,676::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-67::DEBUG::2012-12-09 12:09:07,677::task::588::TaskManager.Task::(_updateState) Task=`fd11a3b1-ce6e-4904-8a07-c06d50134348`::moving from state init -> state preparing
>Thread-67::INFO::2012-12-09 12:09:07,677::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-67::DEBUG::2012-12-09 12:09:07,678::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-67::DEBUG::2012-12-09 12:09:07,678::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-67::INFO::2012-12-09 12:09:07,678::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-67::DEBUG::2012-12-09 12:09:07,679::task::1172::TaskManager.Task::(prepare) Task=`fd11a3b1-ce6e-4904-8a07-c06d50134348`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-67::DEBUG::2012-12-09 12:09:07,679::task::588::TaskManager.Task::(_updateState) Task=`fd11a3b1-ce6e-4904-8a07-c06d50134348`::moving from state preparing -> state finished
>Thread-67::DEBUG::2012-12-09 12:09:07,680::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-67::DEBUG::2012-12-09 12:09:07,680::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-67::DEBUG::2012-12-09 12:09:07,680::task::978::TaskManager.Task::(_decref) Task=`fd11a3b1-ce6e-4904-8a07-c06d50134348`::ref 0 aborting False
>Thread-68::DEBUG::2012-12-09 12:09:08,711::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-68::DEBUG::2012-12-09 12:09:08,712::task::588::TaskManager.Task::(_updateState) Task=`29c5d742-5f82-4c94-ba16-6c47a9b4dadc`::moving from state init -> state preparing
>Thread-68::INFO::2012-12-09 12:09:08,712::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-68::DEBUG::2012-12-09 12:09:08,712::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-68::DEBUG::2012-12-09 12:09:08,712::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-68::INFO::2012-12-09 12:09:08,712::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-68::DEBUG::2012-12-09 12:09:08,713::task::1172::TaskManager.Task::(prepare) Task=`29c5d742-5f82-4c94-ba16-6c47a9b4dadc`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-68::DEBUG::2012-12-09 12:09:08,713::task::588::TaskManager.Task::(_updateState) Task=`29c5d742-5f82-4c94-ba16-6c47a9b4dadc`::moving from state preparing -> state finished
>Thread-68::DEBUG::2012-12-09 12:09:08,713::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-68::DEBUG::2012-12-09 12:09:08,713::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-68::DEBUG::2012-12-09 12:09:08,713::task::978::TaskManager.Task::(_decref) Task=`29c5d742-5f82-4c94-ba16-6c47a9b4dadc`::ref 0 aborting False
>Thread-70::DEBUG::2012-12-09 12:09:09,743::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-70::DEBUG::2012-12-09 12:09:09,744::task::588::TaskManager.Task::(_updateState) Task=`6ab7e3a3-cd31-4b38-ad6f-f05b3ddad07d`::moving from state init -> state preparing
>Thread-70::INFO::2012-12-09 12:09:09,744::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-70::DEBUG::2012-12-09 12:09:09,745::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-70::DEBUG::2012-12-09 12:09:09,745::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-70::INFO::2012-12-09 12:09:09,746::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-70::DEBUG::2012-12-09 12:09:09,746::task::1172::TaskManager.Task::(prepare) Task=`6ab7e3a3-cd31-4b38-ad6f-f05b3ddad07d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-70::DEBUG::2012-12-09 12:09:09,746::task::588::TaskManager.Task::(_updateState) Task=`6ab7e3a3-cd31-4b38-ad6f-f05b3ddad07d`::moving from state preparing -> state finished
>Thread-70::DEBUG::2012-12-09 12:09:09,747::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-70::DEBUG::2012-12-09 12:09:09,747::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-70::DEBUG::2012-12-09 12:09:09,748::task::978::TaskManager.Task::(_decref) Task=`6ab7e3a3-cd31-4b38-ad6f-f05b3ddad07d`::ref 0 aborting False
>Thread-71::DEBUG::2012-12-09 12:09:10,770::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-71::DEBUG::2012-12-09 12:09:10,771::task::588::TaskManager.Task::(_updateState) Task=`c2a51b35-8b3d-4abe-b5a6-0458897614bb`::moving from state init -> state preparing
>Thread-71::INFO::2012-12-09 12:09:10,771::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-71::DEBUG::2012-12-09 12:09:10,771::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-71::DEBUG::2012-12-09 12:09:10,771::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-71::INFO::2012-12-09 12:09:10,772::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-71::DEBUG::2012-12-09 12:09:10,772::task::1172::TaskManager.Task::(prepare) Task=`c2a51b35-8b3d-4abe-b5a6-0458897614bb`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-71::DEBUG::2012-12-09 12:09:10,772::task::588::TaskManager.Task::(_updateState) Task=`c2a51b35-8b3d-4abe-b5a6-0458897614bb`::moving from state preparing -> state finished
>Thread-71::DEBUG::2012-12-09 12:09:10,772::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-71::DEBUG::2012-12-09 12:09:10,772::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-71::DEBUG::2012-12-09 12:09:10,773::task::978::TaskManager.Task::(_decref) Task=`c2a51b35-8b3d-4abe-b5a6-0458897614bb`::ref 0 aborting False
>Thread-72::DEBUG::2012-12-09 12:09:11,298::task::588::TaskManager.Task::(_updateState) Task=`4b4c9903-c22a-4c5a-aca6-a3d5162a11bb`::moving from state init -> state preparing
>Thread-72::INFO::2012-12-09 12:09:11,298::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-72::INFO::2012-12-09 12:09:11,299::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-72::DEBUG::2012-12-09 12:09:11,299::task::1172::TaskManager.Task::(prepare) Task=`4b4c9903-c22a-4c5a-aca6-a3d5162a11bb`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-72::DEBUG::2012-12-09 12:09:11,299::task::588::TaskManager.Task::(_updateState) Task=`4b4c9903-c22a-4c5a-aca6-a3d5162a11bb`::moving from state preparing -> state finished
>Thread-72::DEBUG::2012-12-09 12:09:11,299::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-72::DEBUG::2012-12-09 12:09:11,300::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-72::DEBUG::2012-12-09 12:09:11,300::task::978::TaskManager.Task::(_decref) Task=`4b4c9903-c22a-4c5a-aca6-a3d5162a11bb`::ref 0 aborting False
>Thread-74::DEBUG::2012-12-09 12:09:11,812::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-74::DEBUG::2012-12-09 12:09:11,813::task::588::TaskManager.Task::(_updateState) Task=`2444b110-a018-42d6-a6c5-51eea7674e67`::moving from state init -> state preparing
>Thread-74::INFO::2012-12-09 12:09:11,813::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-74::DEBUG::2012-12-09 12:09:11,814::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-74::DEBUG::2012-12-09 12:09:11,814::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-74::INFO::2012-12-09 12:09:11,814::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-74::DEBUG::2012-12-09 12:09:11,815::task::1172::TaskManager.Task::(prepare) Task=`2444b110-a018-42d6-a6c5-51eea7674e67`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-74::DEBUG::2012-12-09 12:09:11,815::task::588::TaskManager.Task::(_updateState) Task=`2444b110-a018-42d6-a6c5-51eea7674e67`::moving from state preparing -> state finished
>Thread-74::DEBUG::2012-12-09 12:09:11,816::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-74::DEBUG::2012-12-09 12:09:11,816::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-74::DEBUG::2012-12-09 12:09:11,816::task::978::TaskManager.Task::(_decref) Task=`2444b110-a018-42d6-a6c5-51eea7674e67`::ref 0 aborting False
>Thread-75::DEBUG::2012-12-09 12:09:12,848::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-75::DEBUG::2012-12-09 12:09:12,849::task::588::TaskManager.Task::(_updateState) Task=`ad8fecb1-2c04-42c0-a6e0-8fc22e6e1749`::moving from state init -> state preparing
>Thread-75::INFO::2012-12-09 12:09:12,849::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-75::DEBUG::2012-12-09 12:09:12,849::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-75::DEBUG::2012-12-09 12:09:12,849::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-75::INFO::2012-12-09 12:09:12,850::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-75::DEBUG::2012-12-09 12:09:12,850::task::1172::TaskManager.Task::(prepare) Task=`ad8fecb1-2c04-42c0-a6e0-8fc22e6e1749`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-75::DEBUG::2012-12-09 12:09:12,850::task::588::TaskManager.Task::(_updateState) Task=`ad8fecb1-2c04-42c0-a6e0-8fc22e6e1749`::moving from state preparing -> state finished
>Thread-75::DEBUG::2012-12-09 12:09:12,850::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-75::DEBUG::2012-12-09 12:09:12,850::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-75::DEBUG::2012-12-09 12:09:12,851::task::978::TaskManager.Task::(_decref) Task=`ad8fecb1-2c04-42c0-a6e0-8fc22e6e1749`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:09:13,092::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:09:13,093::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:09:13,142::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:09:13,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000542052 s, 7.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:09:13,188::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:13,189::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:09:13,189::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:09:13,190::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-77::DEBUG::2012-12-09 12:09:13,881::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-77::DEBUG::2012-12-09 12:09:13,882::task::588::TaskManager.Task::(_updateState) Task=`4ca230b4-94cc-4059-9b9a-9b0b33f02e55`::moving from state init -> state preparing
>Thread-77::INFO::2012-12-09 12:09:13,882::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-77::DEBUG::2012-12-09 12:09:13,882::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-77::DEBUG::2012-12-09 12:09:13,882::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-77::INFO::2012-12-09 12:09:13,883::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-77::DEBUG::2012-12-09 12:09:13,883::task::1172::TaskManager.Task::(prepare) Task=`4ca230b4-94cc-4059-9b9a-9b0b33f02e55`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-77::DEBUG::2012-12-09 12:09:13,883::task::588::TaskManager.Task::(_updateState) Task=`4ca230b4-94cc-4059-9b9a-9b0b33f02e55`::moving from state preparing -> state finished
>Thread-77::DEBUG::2012-12-09 12:09:13,883::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-77::DEBUG::2012-12-09 12:09:13,883::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-77::DEBUG::2012-12-09 12:09:13,884::task::978::TaskManager.Task::(_decref) Task=`4ca230b4-94cc-4059-9b9a-9b0b33f02e55`::ref 0 aborting False
>Thread-78::DEBUG::2012-12-09 12:09:14,908::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-78::DEBUG::2012-12-09 12:09:14,908::task::588::TaskManager.Task::(_updateState) Task=`816f5dee-abe4-4782-a832-e9bc3ec0b72e`::moving from state init -> state preparing
>Thread-78::INFO::2012-12-09 12:09:14,909::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-78::DEBUG::2012-12-09 12:09:14,909::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-78::DEBUG::2012-12-09 12:09:14,909::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-78::INFO::2012-12-09 12:09:14,909::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-78::DEBUG::2012-12-09 12:09:14,909::task::1172::TaskManager.Task::(prepare) Task=`816f5dee-abe4-4782-a832-e9bc3ec0b72e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-78::DEBUG::2012-12-09 12:09:14,910::task::588::TaskManager.Task::(_updateState) Task=`816f5dee-abe4-4782-a832-e9bc3ec0b72e`::moving from state preparing -> state finished
>Thread-78::DEBUG::2012-12-09 12:09:14,910::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-78::DEBUG::2012-12-09 12:09:14,910::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-78::DEBUG::2012-12-09 12:09:14,910::task::978::TaskManager.Task::(_decref) Task=`816f5dee-abe4-4782-a832-e9bc3ec0b72e`::ref 0 aborting False
>Thread-80::DEBUG::2012-12-09 12:09:15,944::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-80::DEBUG::2012-12-09 12:09:15,945::task::588::TaskManager.Task::(_updateState) Task=`548fe7d5-8a50-4146-a825-bc5cb57c2fa8`::moving from state init -> state preparing
>Thread-80::INFO::2012-12-09 12:09:15,945::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-80::DEBUG::2012-12-09 12:09:15,945::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-80::DEBUG::2012-12-09 12:09:15,946::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-80::INFO::2012-12-09 12:09:15,946::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-80::DEBUG::2012-12-09 12:09:15,946::task::1172::TaskManager.Task::(prepare) Task=`548fe7d5-8a50-4146-a825-bc5cb57c2fa8`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-80::DEBUG::2012-12-09 12:09:15,946::task::588::TaskManager.Task::(_updateState) Task=`548fe7d5-8a50-4146-a825-bc5cb57c2fa8`::moving from state preparing -> state finished
>Thread-80::DEBUG::2012-12-09 12:09:15,946::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-80::DEBUG::2012-12-09 12:09:15,946::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-80::DEBUG::2012-12-09 12:09:15,947::task::978::TaskManager.Task::(_decref) Task=`548fe7d5-8a50-4146-a825-bc5cb57c2fa8`::ref 0 aborting False
>Thread-81::DEBUG::2012-12-09 12:09:16,976::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-81::DEBUG::2012-12-09 12:09:16,977::task::588::TaskManager.Task::(_updateState) Task=`3ecfb138-14f6-434e-95f4-f5ae4185523e`::moving from state init -> state preparing
>Thread-81::INFO::2012-12-09 12:09:16,977::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-81::DEBUG::2012-12-09 12:09:16,978::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-81::DEBUG::2012-12-09 12:09:16,978::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-81::INFO::2012-12-09 12:09:16,978::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-81::DEBUG::2012-12-09 12:09:16,979::task::1172::TaskManager.Task::(prepare) Task=`3ecfb138-14f6-434e-95f4-f5ae4185523e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-81::DEBUG::2012-12-09 12:09:16,979::task::588::TaskManager.Task::(_updateState) Task=`3ecfb138-14f6-434e-95f4-f5ae4185523e`::moving from state preparing -> state finished
>Thread-81::DEBUG::2012-12-09 12:09:16,980::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-81::DEBUG::2012-12-09 12:09:16,980::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-81::DEBUG::2012-12-09 12:09:16,980::task::978::TaskManager.Task::(_decref) Task=`3ecfb138-14f6-434e-95f4-f5ae4185523e`::ref 0 aborting False
>Thread-83::DEBUG::2012-12-09 12:09:18,005::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-83::DEBUG::2012-12-09 12:09:18,006::task::588::TaskManager.Task::(_updateState) Task=`d7950be3-dceb-46d5-ac3d-109a324cbf58`::moving from state init -> state preparing
>Thread-83::INFO::2012-12-09 12:09:18,007::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-83::DEBUG::2012-12-09 12:09:18,007::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-83::DEBUG::2012-12-09 12:09:18,007::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-83::INFO::2012-12-09 12:09:18,008::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-83::DEBUG::2012-12-09 12:09:18,008::task::1172::TaskManager.Task::(prepare) Task=`d7950be3-dceb-46d5-ac3d-109a324cbf58`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-83::DEBUG::2012-12-09 12:09:18,009::task::588::TaskManager.Task::(_updateState) Task=`d7950be3-dceb-46d5-ac3d-109a324cbf58`::moving from state preparing -> state finished
>Thread-83::DEBUG::2012-12-09 12:09:18,009::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-83::DEBUG::2012-12-09 12:09:18,009::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-83::DEBUG::2012-12-09 12:09:18,010::task::978::TaskManager.Task::(_decref) Task=`d7950be3-dceb-46d5-ac3d-109a324cbf58`::ref 0 aborting False
>Thread-84::DEBUG::2012-12-09 12:09:19,036::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-84::DEBUG::2012-12-09 12:09:19,036::task::588::TaskManager.Task::(_updateState) Task=`a4013066-a592-4430-a77e-eca75ba37a1c`::moving from state init -> state preparing
>Thread-84::INFO::2012-12-09 12:09:19,037::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-84::DEBUG::2012-12-09 12:09:19,037::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-84::DEBUG::2012-12-09 12:09:19,038::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-84::INFO::2012-12-09 12:09:19,038::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-84::DEBUG::2012-12-09 12:09:19,038::task::1172::TaskManager.Task::(prepare) Task=`a4013066-a592-4430-a77e-eca75ba37a1c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-84::DEBUG::2012-12-09 12:09:19,039::task::588::TaskManager.Task::(_updateState) Task=`a4013066-a592-4430-a77e-eca75ba37a1c`::moving from state preparing -> state finished
>Thread-84::DEBUG::2012-12-09 12:09:19,039::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-84::DEBUG::2012-12-09 12:09:19,039::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-84::DEBUG::2012-12-09 12:09:19,040::task::978::TaskManager.Task::(_decref) Task=`a4013066-a592-4430-a77e-eca75ba37a1c`::ref 0 aborting False
>Thread-86::DEBUG::2012-12-09 12:09:20,062::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-86::DEBUG::2012-12-09 12:09:20,063::task::588::TaskManager.Task::(_updateState) Task=`36f0bdb6-9c77-4d06-adf2-6eb1e6e7eed8`::moving from state init -> state preparing
>Thread-86::INFO::2012-12-09 12:09:20,063::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-86::DEBUG::2012-12-09 12:09:20,063::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-86::DEBUG::2012-12-09 12:09:20,064::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-86::INFO::2012-12-09 12:09:20,064::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-86::DEBUG::2012-12-09 12:09:20,064::task::1172::TaskManager.Task::(prepare) Task=`36f0bdb6-9c77-4d06-adf2-6eb1e6e7eed8`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-86::DEBUG::2012-12-09 12:09:20,064::task::588::TaskManager.Task::(_updateState) Task=`36f0bdb6-9c77-4d06-adf2-6eb1e6e7eed8`::moving from state preparing -> state finished
>Thread-86::DEBUG::2012-12-09 12:09:20,064::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-86::DEBUG::2012-12-09 12:09:20,065::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-86::DEBUG::2012-12-09 12:09:20,065::task::978::TaskManager.Task::(_decref) Task=`36f0bdb6-9c77-4d06-adf2-6eb1e6e7eed8`::ref 0 aborting False
>Thread-87::DEBUG::2012-12-09 12:09:21,102::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-87::DEBUG::2012-12-09 12:09:21,103::task::588::TaskManager.Task::(_updateState) Task=`689df0f9-ef46-4b61-b1e8-60b6a5c03a98`::moving from state init -> state preparing
>Thread-87::INFO::2012-12-09 12:09:21,104::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-87::DEBUG::2012-12-09 12:09:21,104::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-87::DEBUG::2012-12-09 12:09:21,104::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-87::INFO::2012-12-09 12:09:21,105::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-87::DEBUG::2012-12-09 12:09:21,105::task::1172::TaskManager.Task::(prepare) Task=`689df0f9-ef46-4b61-b1e8-60b6a5c03a98`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-87::DEBUG::2012-12-09 12:09:21,105::task::588::TaskManager.Task::(_updateState) Task=`689df0f9-ef46-4b61-b1e8-60b6a5c03a98`::moving from state preparing -> state finished
>Thread-87::DEBUG::2012-12-09 12:09:21,106::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-87::DEBUG::2012-12-09 12:09:21,106::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-87::DEBUG::2012-12-09 12:09:21,106::task::978::TaskManager.Task::(_decref) Task=`689df0f9-ef46-4b61-b1e8-60b6a5c03a98`::ref 0 aborting False
>Thread-88::DEBUG::2012-12-09 12:09:21,578::task::588::TaskManager.Task::(_updateState) Task=`a299f5af-cb4d-47f8-a8af-70ee366ad58a`::moving from state init -> state preparing
>Thread-88::INFO::2012-12-09 12:09:21,578::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-88::INFO::2012-12-09 12:09:21,579::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '18.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-88::DEBUG::2012-12-09 12:09:21,579::task::1172::TaskManager.Task::(prepare) Task=`a299f5af-cb4d-47f8-a8af-70ee366ad58a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '18.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-88::DEBUG::2012-12-09 12:09:21,579::task::588::TaskManager.Task::(_updateState) Task=`a299f5af-cb4d-47f8-a8af-70ee366ad58a`::moving from state preparing -> state finished
>Thread-88::DEBUG::2012-12-09 12:09:21,580::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-88::DEBUG::2012-12-09 12:09:21,580::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-88::DEBUG::2012-12-09 12:09:21,580::task::978::TaskManager.Task::(_decref) Task=`a299f5af-cb4d-47f8-a8af-70ee366ad58a`::ref 0 aborting False
>Thread-90::DEBUG::2012-12-09 12:09:22,143::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-90::DEBUG::2012-12-09 12:09:22,144::task::588::TaskManager.Task::(_updateState) Task=`a36c594a-2c69-416f-b9e0-ad8c04e2cdb4`::moving from state init -> state preparing
>Thread-90::INFO::2012-12-09 12:09:22,145::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-90::DEBUG::2012-12-09 12:09:22,145::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-90::DEBUG::2012-12-09 12:09:22,146::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-90::INFO::2012-12-09 12:09:22,146::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-90::DEBUG::2012-12-09 12:09:22,147::task::1172::TaskManager.Task::(prepare) Task=`a36c594a-2c69-416f-b9e0-ad8c04e2cdb4`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-90::DEBUG::2012-12-09 12:09:22,147::task::588::TaskManager.Task::(_updateState) Task=`a36c594a-2c69-416f-b9e0-ad8c04e2cdb4`::moving from state preparing -> state finished
>Thread-90::DEBUG::2012-12-09 12:09:22,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-90::DEBUG::2012-12-09 12:09:22,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-90::DEBUG::2012-12-09 12:09:22,148::task::978::TaskManager.Task::(_decref) Task=`a36c594a-2c69-416f-b9e0-ad8c04e2cdb4`::ref 0 aborting False
>Thread-91::DEBUG::2012-12-09 12:09:23,184::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-91::DEBUG::2012-12-09 12:09:23,185::task::588::TaskManager.Task::(_updateState) Task=`0d57d9ff-17e1-4070-8c39-3caf525564e9`::moving from state init -> state preparing
>Thread-91::INFO::2012-12-09 12:09:23,186::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-91::DEBUG::2012-12-09 12:09:23,186::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-91::DEBUG::2012-12-09 12:09:23,186::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-91::INFO::2012-12-09 12:09:23,187::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-91::DEBUG::2012-12-09 12:09:23,187::task::1172::TaskManager.Task::(prepare) Task=`0d57d9ff-17e1-4070-8c39-3caf525564e9`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-91::DEBUG::2012-12-09 12:09:23,188::task::588::TaskManager.Task::(_updateState) Task=`0d57d9ff-17e1-4070-8c39-3caf525564e9`::moving from state preparing -> state finished
>Thread-91::DEBUG::2012-12-09 12:09:23,188::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-91::DEBUG::2012-12-09 12:09:23,188::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-91::DEBUG::2012-12-09 12:09:23,188::task::978::TaskManager.Task::(_decref) Task=`0d57d9ff-17e1-4070-8c39-3caf525564e9`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:09:23,191::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:09:23,191::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:23,298::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:23,300::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:09:23,300::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:09:23,301::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-93::DEBUG::2012-12-09 12:09:24,219::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-93::DEBUG::2012-12-09 12:09:24,219::task::588::TaskManager.Task::(_updateState) Task=`45920b6a-376f-4e55-b34d-cd3258cf6b0b`::moving from state init -> state preparing
>Thread-93::INFO::2012-12-09 12:09:24,220::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-93::DEBUG::2012-12-09 12:09:24,220::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-93::DEBUG::2012-12-09 12:09:24,220::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-93::INFO::2012-12-09 12:09:24,220::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-93::DEBUG::2012-12-09 12:09:24,220::task::1172::TaskManager.Task::(prepare) Task=`45920b6a-376f-4e55-b34d-cd3258cf6b0b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-93::DEBUG::2012-12-09 12:09:24,221::task::588::TaskManager.Task::(_updateState) Task=`45920b6a-376f-4e55-b34d-cd3258cf6b0b`::moving from state preparing -> state finished
>Thread-93::DEBUG::2012-12-09 12:09:24,221::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-93::DEBUG::2012-12-09 12:09:24,221::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-93::DEBUG::2012-12-09 12:09:24,221::task::978::TaskManager.Task::(_decref) Task=`45920b6a-376f-4e55-b34d-cd3258cf6b0b`::ref 0 aborting False
>Thread-94::DEBUG::2012-12-09 12:09:25,256::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-94::DEBUG::2012-12-09 12:09:25,256::task::588::TaskManager.Task::(_updateState) Task=`6bdc84bf-bcc8-4d5b-af6a-298fa719c4ee`::moving from state init -> state preparing
>Thread-94::INFO::2012-12-09 12:09:25,257::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-94::DEBUG::2012-12-09 12:09:25,257::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-94::DEBUG::2012-12-09 12:09:25,258::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-94::INFO::2012-12-09 12:09:25,258::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-94::DEBUG::2012-12-09 12:09:25,258::task::1172::TaskManager.Task::(prepare) Task=`6bdc84bf-bcc8-4d5b-af6a-298fa719c4ee`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-94::DEBUG::2012-12-09 12:09:25,259::task::588::TaskManager.Task::(_updateState) Task=`6bdc84bf-bcc8-4d5b-af6a-298fa719c4ee`::moving from state preparing -> state finished
>Thread-94::DEBUG::2012-12-09 12:09:25,259::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-94::DEBUG::2012-12-09 12:09:25,260::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-94::DEBUG::2012-12-09 12:09:25,260::task::978::TaskManager.Task::(_decref) Task=`6bdc84bf-bcc8-4d5b-af6a-298fa719c4ee`::ref 0 aborting False
>Thread-96::DEBUG::2012-12-09 12:09:26,287::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-96::DEBUG::2012-12-09 12:09:26,287::task::588::TaskManager.Task::(_updateState) Task=`d9ada5ba-5a99-49eb-b508-a727404db7f3`::moving from state init -> state preparing
>Thread-96::INFO::2012-12-09 12:09:26,288::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-96::DEBUG::2012-12-09 12:09:26,288::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-96::DEBUG::2012-12-09 12:09:26,288::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-96::INFO::2012-12-09 12:09:26,288::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-96::DEBUG::2012-12-09 12:09:26,288::task::1172::TaskManager.Task::(prepare) Task=`d9ada5ba-5a99-49eb-b508-a727404db7f3`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-96::DEBUG::2012-12-09 12:09:26,288::task::588::TaskManager.Task::(_updateState) Task=`d9ada5ba-5a99-49eb-b508-a727404db7f3`::moving from state preparing -> state finished
>Thread-96::DEBUG::2012-12-09 12:09:26,289::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-96::DEBUG::2012-12-09 12:09:26,289::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-96::DEBUG::2012-12-09 12:09:26,289::task::978::TaskManager.Task::(_decref) Task=`d9ada5ba-5a99-49eb-b508-a727404db7f3`::ref 0 aborting False
>Thread-97::DEBUG::2012-12-09 12:09:27,321::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-97::DEBUG::2012-12-09 12:09:27,322::task::588::TaskManager.Task::(_updateState) Task=`696cd809-8a0c-4b12-9ec1-d597c76a3971`::moving from state init -> state preparing
>Thread-97::INFO::2012-12-09 12:09:27,322::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-97::DEBUG::2012-12-09 12:09:27,322::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-97::DEBUG::2012-12-09 12:09:27,322::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-97::INFO::2012-12-09 12:09:27,322::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-97::DEBUG::2012-12-09 12:09:27,323::task::1172::TaskManager.Task::(prepare) Task=`696cd809-8a0c-4b12-9ec1-d597c76a3971`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-97::DEBUG::2012-12-09 12:09:27,323::task::588::TaskManager.Task::(_updateState) Task=`696cd809-8a0c-4b12-9ec1-d597c76a3971`::moving from state preparing -> state finished
>Thread-97::DEBUG::2012-12-09 12:09:27,323::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-97::DEBUG::2012-12-09 12:09:27,323::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-97::DEBUG::2012-12-09 12:09:27,323::task::978::TaskManager.Task::(_decref) Task=`696cd809-8a0c-4b12-9ec1-d597c76a3971`::ref 0 aborting False
>Thread-99::DEBUG::2012-12-09 12:09:28,352::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-99::DEBUG::2012-12-09 12:09:28,353::task::588::TaskManager.Task::(_updateState) Task=`bda7631e-e171-407f-a41f-087d55ff489f`::moving from state init -> state preparing
>Thread-99::INFO::2012-12-09 12:09:28,353::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-99::DEBUG::2012-12-09 12:09:28,353::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-99::DEBUG::2012-12-09 12:09:28,353::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-99::INFO::2012-12-09 12:09:28,353::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-99::DEBUG::2012-12-09 12:09:28,354::task::1172::TaskManager.Task::(prepare) Task=`bda7631e-e171-407f-a41f-087d55ff489f`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-99::DEBUG::2012-12-09 12:09:28,354::task::588::TaskManager.Task::(_updateState) Task=`bda7631e-e171-407f-a41f-087d55ff489f`::moving from state preparing -> state finished
>Thread-99::DEBUG::2012-12-09 12:09:28,355::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-99::DEBUG::2012-12-09 12:09:28,355::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-99::DEBUG::2012-12-09 12:09:28,355::task::978::TaskManager.Task::(_decref) Task=`bda7631e-e171-407f-a41f-087d55ff489f`::ref 0 aborting False
>Thread-100::DEBUG::2012-12-09 12:09:29,392::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-100::DEBUG::2012-12-09 12:09:29,393::task::588::TaskManager.Task::(_updateState) Task=`b0e9a69a-3541-4557-9339-d4c05685bb8b`::moving from state init -> state preparing
>Thread-100::INFO::2012-12-09 12:09:29,394::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-100::DEBUG::2012-12-09 12:09:29,394::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-100::DEBUG::2012-12-09 12:09:29,395::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-100::INFO::2012-12-09 12:09:29,395::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-100::DEBUG::2012-12-09 12:09:29,395::task::1172::TaskManager.Task::(prepare) Task=`b0e9a69a-3541-4557-9339-d4c05685bb8b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-100::DEBUG::2012-12-09 12:09:29,396::task::588::TaskManager.Task::(_updateState) Task=`b0e9a69a-3541-4557-9339-d4c05685bb8b`::moving from state preparing -> state finished
>Thread-100::DEBUG::2012-12-09 12:09:29,396::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-100::DEBUG::2012-12-09 12:09:29,397::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-100::DEBUG::2012-12-09 12:09:29,397::task::978::TaskManager.Task::(_decref) Task=`b0e9a69a-3541-4557-9339-d4c05685bb8b`::ref 0 aborting False
>Thread-102::DEBUG::2012-12-09 12:09:30,439::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-102::DEBUG::2012-12-09 12:09:30,439::task::588::TaskManager.Task::(_updateState) Task=`4ceabe83-09c7-4a2f-8fac-a5e8e3d8780e`::moving from state init -> state preparing
>Thread-102::INFO::2012-12-09 12:09:30,439::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-102::DEBUG::2012-12-09 12:09:30,440::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-102::DEBUG::2012-12-09 12:09:30,440::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-102::INFO::2012-12-09 12:09:30,440::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-102::DEBUG::2012-12-09 12:09:30,440::task::1172::TaskManager.Task::(prepare) Task=`4ceabe83-09c7-4a2f-8fac-a5e8e3d8780e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-102::DEBUG::2012-12-09 12:09:30,440::task::588::TaskManager.Task::(_updateState) Task=`4ceabe83-09c7-4a2f-8fac-a5e8e3d8780e`::moving from state preparing -> state finished
>Thread-102::DEBUG::2012-12-09 12:09:30,441::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-102::DEBUG::2012-12-09 12:09:30,441::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-102::DEBUG::2012-12-09 12:09:30,441::task::978::TaskManager.Task::(_decref) Task=`4ceabe83-09c7-4a2f-8fac-a5e8e3d8780e`::ref 0 aborting False
>Thread-103::DEBUG::2012-12-09 12:09:31,468::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-103::DEBUG::2012-12-09 12:09:31,468::task::588::TaskManager.Task::(_updateState) Task=`57019662-1842-480b-8f68-76baf93b606b`::moving from state init -> state preparing
>Thread-103::INFO::2012-12-09 12:09:31,468::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-103::DEBUG::2012-12-09 12:09:31,469::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-103::DEBUG::2012-12-09 12:09:31,469::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-103::INFO::2012-12-09 12:09:31,469::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-103::DEBUG::2012-12-09 12:09:31,469::task::1172::TaskManager.Task::(prepare) Task=`57019662-1842-480b-8f68-76baf93b606b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-103::DEBUG::2012-12-09 12:09:31,469::task::588::TaskManager.Task::(_updateState) Task=`57019662-1842-480b-8f68-76baf93b606b`::moving from state preparing -> state finished
>Thread-103::DEBUG::2012-12-09 12:09:31,470::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-103::DEBUG::2012-12-09 12:09:31,470::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-103::DEBUG::2012-12-09 12:09:31,470::task::978::TaskManager.Task::(_decref) Task=`57019662-1842-480b-8f68-76baf93b606b`::ref 0 aborting False
>Thread-104::DEBUG::2012-12-09 12:09:31,843::task::588::TaskManager.Task::(_updateState) Task=`329274fc-4903-44aa-a650-27a997cb0a9e`::moving from state init -> state preparing
>Thread-104::INFO::2012-12-09 12:09:31,844::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-104::INFO::2012-12-09 12:09:31,844::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '28.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-104::DEBUG::2012-12-09 12:09:31,844::task::1172::TaskManager.Task::(prepare) Task=`329274fc-4903-44aa-a650-27a997cb0a9e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '28.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-104::DEBUG::2012-12-09 12:09:31,844::task::588::TaskManager.Task::(_updateState) Task=`329274fc-4903-44aa-a650-27a997cb0a9e`::moving from state preparing -> state finished
>Thread-104::DEBUG::2012-12-09 12:09:31,844::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-104::DEBUG::2012-12-09 12:09:31,845::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-104::DEBUG::2012-12-09 12:09:31,845::task::978::TaskManager.Task::(_decref) Task=`329274fc-4903-44aa-a650-27a997cb0a9e`::ref 0 aborting False
>Thread-106::DEBUG::2012-12-09 12:09:32,506::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-106::DEBUG::2012-12-09 12:09:32,507::task::588::TaskManager.Task::(_updateState) Task=`da779fe7-d68a-47f6-972f-ef3eee2b7038`::moving from state init -> state preparing
>Thread-106::INFO::2012-12-09 12:09:32,507::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-106::DEBUG::2012-12-09 12:09:32,508::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-106::DEBUG::2012-12-09 12:09:32,508::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-106::INFO::2012-12-09 12:09:32,508::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-106::DEBUG::2012-12-09 12:09:32,509::task::1172::TaskManager.Task::(prepare) Task=`da779fe7-d68a-47f6-972f-ef3eee2b7038`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-106::DEBUG::2012-12-09 12:09:32,509::task::588::TaskManager.Task::(_updateState) Task=`da779fe7-d68a-47f6-972f-ef3eee2b7038`::moving from state preparing -> state finished
>Thread-106::DEBUG::2012-12-09 12:09:32,510::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-106::DEBUG::2012-12-09 12:09:32,510::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-106::DEBUG::2012-12-09 12:09:32,510::task::978::TaskManager.Task::(_decref) Task=`da779fe7-d68a-47f6-972f-ef3eee2b7038`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:09:33,301::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:09:33,302::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:33,418::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:33,420::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:09:33,420::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:09:33,420::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-107::DEBUG::2012-12-09 12:09:33,540::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-107::DEBUG::2012-12-09 12:09:33,541::task::588::TaskManager.Task::(_updateState) Task=`fe6d7c69-60ae-43f0-9224-1bc637efa8b5`::moving from state init -> state preparing
>Thread-107::INFO::2012-12-09 12:09:33,541::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-107::DEBUG::2012-12-09 12:09:33,541::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-107::DEBUG::2012-12-09 12:09:33,541::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-107::INFO::2012-12-09 12:09:33,541::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-107::DEBUG::2012-12-09 12:09:33,542::task::1172::TaskManager.Task::(prepare) Task=`fe6d7c69-60ae-43f0-9224-1bc637efa8b5`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-107::DEBUG::2012-12-09 12:09:33,542::task::588::TaskManager.Task::(_updateState) Task=`fe6d7c69-60ae-43f0-9224-1bc637efa8b5`::moving from state preparing -> state finished
>Thread-107::DEBUG::2012-12-09 12:09:33,542::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-107::DEBUG::2012-12-09 12:09:33,542::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-107::DEBUG::2012-12-09 12:09:33,542::task::978::TaskManager.Task::(_decref) Task=`fe6d7c69-60ae-43f0-9224-1bc637efa8b5`::ref 0 aborting False
>Thread-109::DEBUG::2012-12-09 12:09:34,573::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-109::DEBUG::2012-12-09 12:09:34,574::task::588::TaskManager.Task::(_updateState) Task=`19a8fbd3-ea51-45ff-bcf1-676faae48418`::moving from state init -> state preparing
>Thread-109::INFO::2012-12-09 12:09:34,574::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-109::DEBUG::2012-12-09 12:09:34,575::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-109::DEBUG::2012-12-09 12:09:34,575::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-109::INFO::2012-12-09 12:09:34,575::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-109::DEBUG::2012-12-09 12:09:34,575::task::1172::TaskManager.Task::(prepare) Task=`19a8fbd3-ea51-45ff-bcf1-676faae48418`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-109::DEBUG::2012-12-09 12:09:34,576::task::588::TaskManager.Task::(_updateState) Task=`19a8fbd3-ea51-45ff-bcf1-676faae48418`::moving from state preparing -> state finished
>Thread-109::DEBUG::2012-12-09 12:09:34,576::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-109::DEBUG::2012-12-09 12:09:34,576::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-109::DEBUG::2012-12-09 12:09:34,576::task::978::TaskManager.Task::(_decref) Task=`19a8fbd3-ea51-45ff-bcf1-676faae48418`::ref 0 aborting False
>Thread-110::DEBUG::2012-12-09 12:09:35,607::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-110::DEBUG::2012-12-09 12:09:35,608::task::588::TaskManager.Task::(_updateState) Task=`21356996-5b2d-4c83-9c15-55568ea2df95`::moving from state init -> state preparing
>Thread-110::INFO::2012-12-09 12:09:35,608::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-110::DEBUG::2012-12-09 12:09:35,609::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-110::DEBUG::2012-12-09 12:09:35,609::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-110::INFO::2012-12-09 12:09:35,609::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-110::DEBUG::2012-12-09 12:09:35,610::task::1172::TaskManager.Task::(prepare) Task=`21356996-5b2d-4c83-9c15-55568ea2df95`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-110::DEBUG::2012-12-09 12:09:35,610::task::588::TaskManager.Task::(_updateState) Task=`21356996-5b2d-4c83-9c15-55568ea2df95`::moving from state preparing -> state finished
>Thread-110::DEBUG::2012-12-09 12:09:35,611::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-110::DEBUG::2012-12-09 12:09:35,611::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-110::DEBUG::2012-12-09 12:09:35,611::task::978::TaskManager.Task::(_decref) Task=`21356996-5b2d-4c83-9c15-55568ea2df95`::ref 0 aborting False
>Thread-112::DEBUG::2012-12-09 12:09:36,640::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-112::DEBUG::2012-12-09 12:09:36,641::task::588::TaskManager.Task::(_updateState) Task=`a4b03a40-ede6-4ec2-9e10-c0787022bc1c`::moving from state init -> state preparing
>Thread-112::INFO::2012-12-09 12:09:36,641::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-112::DEBUG::2012-12-09 12:09:36,641::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-112::DEBUG::2012-12-09 12:09:36,642::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-112::INFO::2012-12-09 12:09:36,642::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-112::DEBUG::2012-12-09 12:09:36,643::task::1172::TaskManager.Task::(prepare) Task=`a4b03a40-ede6-4ec2-9e10-c0787022bc1c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-112::DEBUG::2012-12-09 12:09:36,643::task::588::TaskManager.Task::(_updateState) Task=`a4b03a40-ede6-4ec2-9e10-c0787022bc1c`::moving from state preparing -> state finished
>Thread-112::DEBUG::2012-12-09 12:09:36,644::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-112::DEBUG::2012-12-09 12:09:36,644::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-112::DEBUG::2012-12-09 12:09:36,644::task::978::TaskManager.Task::(_decref) Task=`a4b03a40-ede6-4ec2-9e10-c0787022bc1c`::ref 0 aborting False
>Thread-113::DEBUG::2012-12-09 12:09:37,671::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-113::DEBUG::2012-12-09 12:09:37,671::task::588::TaskManager.Task::(_updateState) Task=`583875e4-e28d-4cd8-8a8a-569db9aadaf5`::moving from state init -> state preparing
>Thread-113::INFO::2012-12-09 12:09:37,671::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-113::DEBUG::2012-12-09 12:09:37,671::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-113::DEBUG::2012-12-09 12:09:37,672::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-113::INFO::2012-12-09 12:09:37,672::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-113::DEBUG::2012-12-09 12:09:37,672::task::1172::TaskManager.Task::(prepare) Task=`583875e4-e28d-4cd8-8a8a-569db9aadaf5`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-113::DEBUG::2012-12-09 12:09:37,672::task::588::TaskManager.Task::(_updateState) Task=`583875e4-e28d-4cd8-8a8a-569db9aadaf5`::moving from state preparing -> state finished
>Thread-113::DEBUG::2012-12-09 12:09:37,672::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-113::DEBUG::2012-12-09 12:09:37,673::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-113::DEBUG::2012-12-09 12:09:37,673::task::978::TaskManager.Task::(_decref) Task=`583875e4-e28d-4cd8-8a8a-569db9aadaf5`::ref 0 aborting False
>Thread-115::DEBUG::2012-12-09 12:09:38,710::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-115::DEBUG::2012-12-09 12:09:38,711::task::588::TaskManager.Task::(_updateState) Task=`b967ff3d-9cb9-4ab7-bdbd-497148d8328f`::moving from state init -> state preparing
>Thread-115::INFO::2012-12-09 12:09:38,711::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-115::DEBUG::2012-12-09 12:09:38,711::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-115::DEBUG::2012-12-09 12:09:38,712::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-115::INFO::2012-12-09 12:09:38,712::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-115::DEBUG::2012-12-09 12:09:38,712::task::1172::TaskManager.Task::(prepare) Task=`b967ff3d-9cb9-4ab7-bdbd-497148d8328f`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-115::DEBUG::2012-12-09 12:09:38,712::task::588::TaskManager.Task::(_updateState) Task=`b967ff3d-9cb9-4ab7-bdbd-497148d8328f`::moving from state preparing -> state finished
>Thread-115::DEBUG::2012-12-09 12:09:38,712::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-115::DEBUG::2012-12-09 12:09:38,712::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-115::DEBUG::2012-12-09 12:09:38,713::task::978::TaskManager.Task::(_decref) Task=`b967ff3d-9cb9-4ab7-bdbd-497148d8328f`::ref 0 aborting False
>Thread-116::DEBUG::2012-12-09 12:09:39,738::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-116::DEBUG::2012-12-09 12:09:39,740::task::588::TaskManager.Task::(_updateState) Task=`a794488a-2dc1-48c8-b1df-4a07b3a14a47`::moving from state init -> state preparing
>Thread-116::INFO::2012-12-09 12:09:39,740::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-116::DEBUG::2012-12-09 12:09:39,740::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-116::DEBUG::2012-12-09 12:09:39,741::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-116::INFO::2012-12-09 12:09:39,741::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-116::DEBUG::2012-12-09 12:09:39,742::task::1172::TaskManager.Task::(prepare) Task=`a794488a-2dc1-48c8-b1df-4a07b3a14a47`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-116::DEBUG::2012-12-09 12:09:39,742::task::588::TaskManager.Task::(_updateState) Task=`a794488a-2dc1-48c8-b1df-4a07b3a14a47`::moving from state preparing -> state finished
>Thread-116::DEBUG::2012-12-09 12:09:39,742::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-116::DEBUG::2012-12-09 12:09:39,743::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-116::DEBUG::2012-12-09 12:09:39,743::task::978::TaskManager.Task::(_decref) Task=`a794488a-2dc1-48c8-b1df-4a07b3a14a47`::ref 0 aborting False
>Thread-118::DEBUG::2012-12-09 12:09:40,768::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-118::DEBUG::2012-12-09 12:09:40,769::task::588::TaskManager.Task::(_updateState) Task=`4fae3c44-699e-4001-bd8d-97a39b51b909`::moving from state init -> state preparing
>Thread-118::INFO::2012-12-09 12:09:40,769::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-118::DEBUG::2012-12-09 12:09:40,769::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-118::DEBUG::2012-12-09 12:09:40,769::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-118::INFO::2012-12-09 12:09:40,770::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-118::DEBUG::2012-12-09 12:09:40,770::task::1172::TaskManager.Task::(prepare) Task=`4fae3c44-699e-4001-bd8d-97a39b51b909`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-118::DEBUG::2012-12-09 12:09:40,770::task::588::TaskManager.Task::(_updateState) Task=`4fae3c44-699e-4001-bd8d-97a39b51b909`::moving from state preparing -> state finished
>Thread-118::DEBUG::2012-12-09 12:09:40,771::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-118::DEBUG::2012-12-09 12:09:40,771::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-118::DEBUG::2012-12-09 12:09:40,771::task::978::TaskManager.Task::(_decref) Task=`4fae3c44-699e-4001-bd8d-97a39b51b909`::ref 0 aborting False
>Thread-119::DEBUG::2012-12-09 12:09:41,811::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-119::DEBUG::2012-12-09 12:09:41,812::task::588::TaskManager.Task::(_updateState) Task=`a1e752f5-9340-4696-afb0-6c144391f2af`::moving from state init -> state preparing
>Thread-119::INFO::2012-12-09 12:09:41,812::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-119::DEBUG::2012-12-09 12:09:41,813::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-119::DEBUG::2012-12-09 12:09:41,813::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-119::INFO::2012-12-09 12:09:41,814::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-119::DEBUG::2012-12-09 12:09:41,814::task::1172::TaskManager.Task::(prepare) Task=`a1e752f5-9340-4696-afb0-6c144391f2af`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-119::DEBUG::2012-12-09 12:09:41,814::task::588::TaskManager.Task::(_updateState) Task=`a1e752f5-9340-4696-afb0-6c144391f2af`::moving from state preparing -> state finished
>Thread-119::DEBUG::2012-12-09 12:09:41,815::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-119::DEBUG::2012-12-09 12:09:41,815::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-119::DEBUG::2012-12-09 12:09:41,816::task::978::TaskManager.Task::(_decref) Task=`a1e752f5-9340-4696-afb0-6c144391f2af`::ref 0 aborting False
>Thread-120::DEBUG::2012-12-09 12:09:42,112::task::588::TaskManager.Task::(_updateState) Task=`ee2a3c35-73be-47d2-b867-8ec4792bf585`::moving from state init -> state preparing
>Thread-120::INFO::2012-12-09 12:09:42,112::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-120::INFO::2012-12-09 12:09:42,112::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '39.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-120::DEBUG::2012-12-09 12:09:42,112::task::1172::TaskManager.Task::(prepare) Task=`ee2a3c35-73be-47d2-b867-8ec4792bf585`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '39.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-120::DEBUG::2012-12-09 12:09:42,113::task::588::TaskManager.Task::(_updateState) Task=`ee2a3c35-73be-47d2-b867-8ec4792bf585`::moving from state preparing -> state finished
>Thread-120::DEBUG::2012-12-09 12:09:42,113::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-120::DEBUG::2012-12-09 12:09:42,113::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-120::DEBUG::2012-12-09 12:09:42,113::task::978::TaskManager.Task::(_decref) Task=`ee2a3c35-73be-47d2-b867-8ec4792bf585`::ref 0 aborting False
>Thread-122::DEBUG::2012-12-09 12:09:42,841::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-122::DEBUG::2012-12-09 12:09:42,842::task::588::TaskManager.Task::(_updateState) Task=`970671d7-1e02-48bb-8149-cc72cb5eff29`::moving from state init -> state preparing
>Thread-122::INFO::2012-12-09 12:09:42,842::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-122::DEBUG::2012-12-09 12:09:42,843::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-122::DEBUG::2012-12-09 12:09:42,843::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-122::INFO::2012-12-09 12:09:42,844::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-122::DEBUG::2012-12-09 12:09:42,844::task::1172::TaskManager.Task::(prepare) Task=`970671d7-1e02-48bb-8149-cc72cb5eff29`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-122::DEBUG::2012-12-09 12:09:42,844::task::588::TaskManager.Task::(_updateState) Task=`970671d7-1e02-48bb-8149-cc72cb5eff29`::moving from state preparing -> state finished
>Thread-122::DEBUG::2012-12-09 12:09:42,845::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-122::DEBUG::2012-12-09 12:09:42,845::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-122::DEBUG::2012-12-09 12:09:42,846::task::978::TaskManager.Task::(_decref) Task=`970671d7-1e02-48bb-8149-cc72cb5eff29`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:09:43,421::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:09:43,422::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:43,538::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:43,540::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:09:43,540::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:09:43,540::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-123::DEBUG::2012-12-09 12:09:43,869::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-123::DEBUG::2012-12-09 12:09:43,870::task::588::TaskManager.Task::(_updateState) Task=`3845ac7d-cee9-4cca-a9fa-b3861e192794`::moving from state init -> state preparing
>Thread-123::INFO::2012-12-09 12:09:43,871::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-123::DEBUG::2012-12-09 12:09:43,871::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-123::DEBUG::2012-12-09 12:09:43,872::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-123::INFO::2012-12-09 12:09:43,872::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-123::DEBUG::2012-12-09 12:09:43,872::task::1172::TaskManager.Task::(prepare) Task=`3845ac7d-cee9-4cca-a9fa-b3861e192794`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-123::DEBUG::2012-12-09 12:09:43,873::task::588::TaskManager.Task::(_updateState) Task=`3845ac7d-cee9-4cca-a9fa-b3861e192794`::moving from state preparing -> state finished
>Thread-123::DEBUG::2012-12-09 12:09:43,873::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-123::DEBUG::2012-12-09 12:09:43,874::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-123::DEBUG::2012-12-09 12:09:43,874::task::978::TaskManager.Task::(_decref) Task=`3845ac7d-cee9-4cca-a9fa-b3861e192794`::ref 0 aborting False
>Thread-125::DEBUG::2012-12-09 12:09:44,904::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-125::DEBUG::2012-12-09 12:09:44,905::task::588::TaskManager.Task::(_updateState) Task=`ffdb1828-4761-4251-a26d-64573cbcc541`::moving from state init -> state preparing
>Thread-125::INFO::2012-12-09 12:09:44,906::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-125::DEBUG::2012-12-09 12:09:44,906::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-125::DEBUG::2012-12-09 12:09:44,907::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-125::INFO::2012-12-09 12:09:44,907::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-125::DEBUG::2012-12-09 12:09:44,908::task::1172::TaskManager.Task::(prepare) Task=`ffdb1828-4761-4251-a26d-64573cbcc541`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-125::DEBUG::2012-12-09 12:09:44,908::task::588::TaskManager.Task::(_updateState) Task=`ffdb1828-4761-4251-a26d-64573cbcc541`::moving from state preparing -> state finished
>Thread-125::DEBUG::2012-12-09 12:09:44,908::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-125::DEBUG::2012-12-09 12:09:44,909::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-125::DEBUG::2012-12-09 12:09:44,909::task::978::TaskManager.Task::(_decref) Task=`ffdb1828-4761-4251-a26d-64573cbcc541`::ref 0 aborting False
>Thread-126::DEBUG::2012-12-09 12:09:45,939::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-126::DEBUG::2012-12-09 12:09:45,940::task::588::TaskManager.Task::(_updateState) Task=`8e55c2d1-5a35-4d24-9148-86d329173f81`::moving from state init -> state preparing
>Thread-126::INFO::2012-12-09 12:09:45,940::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-126::DEBUG::2012-12-09 12:09:45,940::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-126::DEBUG::2012-12-09 12:09:45,940::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-126::INFO::2012-12-09 12:09:45,940::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-126::DEBUG::2012-12-09 12:09:45,941::task::1172::TaskManager.Task::(prepare) Task=`8e55c2d1-5a35-4d24-9148-86d329173f81`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-126::DEBUG::2012-12-09 12:09:45,941::task::588::TaskManager.Task::(_updateState) Task=`8e55c2d1-5a35-4d24-9148-86d329173f81`::moving from state preparing -> state finished
>Thread-126::DEBUG::2012-12-09 12:09:45,941::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-126::DEBUG::2012-12-09 12:09:45,941::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-126::DEBUG::2012-12-09 12:09:45,941::task::978::TaskManager.Task::(_decref) Task=`8e55c2d1-5a35-4d24-9148-86d329173f81`::ref 0 aborting False
>Thread-128::DEBUG::2012-12-09 12:09:46,973::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-128::DEBUG::2012-12-09 12:09:46,974::task::588::TaskManager.Task::(_updateState) Task=`8355a570-df49-4850-b00c-202a4e566a9d`::moving from state init -> state preparing
>Thread-128::INFO::2012-12-09 12:09:46,974::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-128::DEBUG::2012-12-09 12:09:46,974::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-128::DEBUG::2012-12-09 12:09:46,974::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-128::INFO::2012-12-09 12:09:46,975::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-128::DEBUG::2012-12-09 12:09:46,975::task::1172::TaskManager.Task::(prepare) Task=`8355a570-df49-4850-b00c-202a4e566a9d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-128::DEBUG::2012-12-09 12:09:46,975::task::588::TaskManager.Task::(_updateState) Task=`8355a570-df49-4850-b00c-202a4e566a9d`::moving from state preparing -> state finished
>Thread-128::DEBUG::2012-12-09 12:09:46,975::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-128::DEBUG::2012-12-09 12:09:46,975::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-128::DEBUG::2012-12-09 12:09:46,976::task::978::TaskManager.Task::(_decref) Task=`8355a570-df49-4850-b00c-202a4e566a9d`::ref 0 aborting False
>Thread-129::DEBUG::2012-12-09 12:09:48,013::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-129::DEBUG::2012-12-09 12:09:48,014::task::588::TaskManager.Task::(_updateState) Task=`871a2fbf-b584-4214-9e72-8c4f044d78d8`::moving from state init -> state preparing
>Thread-129::INFO::2012-12-09 12:09:48,015::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-129::DEBUG::2012-12-09 12:09:48,015::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-129::DEBUG::2012-12-09 12:09:48,016::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-129::INFO::2012-12-09 12:09:48,016::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-129::DEBUG::2012-12-09 12:09:48,016::task::1172::TaskManager.Task::(prepare) Task=`871a2fbf-b584-4214-9e72-8c4f044d78d8`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-129::DEBUG::2012-12-09 12:09:48,017::task::588::TaskManager.Task::(_updateState) Task=`871a2fbf-b584-4214-9e72-8c4f044d78d8`::moving from state preparing -> state finished
>Thread-129::DEBUG::2012-12-09 12:09:48,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-129::DEBUG::2012-12-09 12:09:48,018::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-129::DEBUG::2012-12-09 12:09:48,018::task::978::TaskManager.Task::(_decref) Task=`871a2fbf-b584-4214-9e72-8c4f044d78d8`::ref 0 aborting False
>Thread-131::DEBUG::2012-12-09 12:09:49,061::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-131::DEBUG::2012-12-09 12:09:49,062::task::588::TaskManager.Task::(_updateState) Task=`761ab8cc-9d92-41f5-8617-1c770e8dd5ca`::moving from state init -> state preparing
>Thread-131::INFO::2012-12-09 12:09:49,062::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-131::DEBUG::2012-12-09 12:09:49,063::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-131::DEBUG::2012-12-09 12:09:49,063::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-131::INFO::2012-12-09 12:09:49,063::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-131::DEBUG::2012-12-09 12:09:49,064::task::1172::TaskManager.Task::(prepare) Task=`761ab8cc-9d92-41f5-8617-1c770e8dd5ca`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-131::DEBUG::2012-12-09 12:09:49,064::task::588::TaskManager.Task::(_updateState) Task=`761ab8cc-9d92-41f5-8617-1c770e8dd5ca`::moving from state preparing -> state finished
>Thread-131::DEBUG::2012-12-09 12:09:49,064::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-131::DEBUG::2012-12-09 12:09:49,064::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-131::DEBUG::2012-12-09 12:09:49,065::task::978::TaskManager.Task::(_decref) Task=`761ab8cc-9d92-41f5-8617-1c770e8dd5ca`::ref 0 aborting False
>Thread-132::DEBUG::2012-12-09 12:09:50,091::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-132::DEBUG::2012-12-09 12:09:50,092::task::588::TaskManager.Task::(_updateState) Task=`4181105d-f8c0-400c-a6f6-a933a9d79af9`::moving from state init -> state preparing
>Thread-132::INFO::2012-12-09 12:09:50,092::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-132::DEBUG::2012-12-09 12:09:50,093::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-132::DEBUG::2012-12-09 12:09:50,093::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-132::INFO::2012-12-09 12:09:50,093::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-132::DEBUG::2012-12-09 12:09:50,093::task::1172::TaskManager.Task::(prepare) Task=`4181105d-f8c0-400c-a6f6-a933a9d79af9`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-132::DEBUG::2012-12-09 12:09:50,094::task::588::TaskManager.Task::(_updateState) Task=`4181105d-f8c0-400c-a6f6-a933a9d79af9`::moving from state preparing -> state finished
>Thread-132::DEBUG::2012-12-09 12:09:50,094::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-132::DEBUG::2012-12-09 12:09:50,094::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-132::DEBUG::2012-12-09 12:09:50,094::task::978::TaskManager.Task::(_decref) Task=`4181105d-f8c0-400c-a6f6-a933a9d79af9`::ref 0 aborting False
>Thread-134::DEBUG::2012-12-09 12:09:51,131::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-134::DEBUG::2012-12-09 12:09:51,132::task::588::TaskManager.Task::(_updateState) Task=`711ffdee-42cf-46e7-bc08-4ab935eac64c`::moving from state init -> state preparing
>Thread-134::INFO::2012-12-09 12:09:51,132::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-134::DEBUG::2012-12-09 12:09:51,132::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-134::DEBUG::2012-12-09 12:09:51,132::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-134::INFO::2012-12-09 12:09:51,132::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-134::DEBUG::2012-12-09 12:09:51,133::task::1172::TaskManager.Task::(prepare) Task=`711ffdee-42cf-46e7-bc08-4ab935eac64c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-134::DEBUG::2012-12-09 12:09:51,133::task::588::TaskManager.Task::(_updateState) Task=`711ffdee-42cf-46e7-bc08-4ab935eac64c`::moving from state preparing -> state finished
>Thread-134::DEBUG::2012-12-09 12:09:51,133::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-134::DEBUG::2012-12-09 12:09:51,133::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-134::DEBUG::2012-12-09 12:09:51,133::task::978::TaskManager.Task::(_decref) Task=`711ffdee-42cf-46e7-bc08-4ab935eac64c`::ref 0 aborting False
>Thread-135::DEBUG::2012-12-09 12:09:52,155::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-135::DEBUG::2012-12-09 12:09:52,155::task::588::TaskManager.Task::(_updateState) Task=`25844d10-4943-4e21-9dcc-1e2ca1f622b7`::moving from state init -> state preparing
>Thread-135::INFO::2012-12-09 12:09:52,156::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-135::DEBUG::2012-12-09 12:09:52,156::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-135::DEBUG::2012-12-09 12:09:52,156::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-135::INFO::2012-12-09 12:09:52,156::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-135::DEBUG::2012-12-09 12:09:52,156::task::1172::TaskManager.Task::(prepare) Task=`25844d10-4943-4e21-9dcc-1e2ca1f622b7`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-135::DEBUG::2012-12-09 12:09:52,156::task::588::TaskManager.Task::(_updateState) Task=`25844d10-4943-4e21-9dcc-1e2ca1f622b7`::moving from state preparing -> state finished
>Thread-135::DEBUG::2012-12-09 12:09:52,157::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-135::DEBUG::2012-12-09 12:09:52,157::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-135::DEBUG::2012-12-09 12:09:52,157::task::978::TaskManager.Task::(_decref) Task=`25844d10-4943-4e21-9dcc-1e2ca1f622b7`::ref 0 aborting False
>Thread-136::DEBUG::2012-12-09 12:09:52,407::task::588::TaskManager.Task::(_updateState) Task=`f46afdfd-0b8c-4429-9b5b-651a4080ddff`::moving from state init -> state preparing
>Thread-136::INFO::2012-12-09 12:09:52,407::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-136::INFO::2012-12-09 12:09:52,408::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '49.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-136::DEBUG::2012-12-09 12:09:52,408::task::1172::TaskManager.Task::(prepare) Task=`f46afdfd-0b8c-4429-9b5b-651a4080ddff`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '49.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-136::DEBUG::2012-12-09 12:09:52,409::task::588::TaskManager.Task::(_updateState) Task=`f46afdfd-0b8c-4429-9b5b-651a4080ddff`::moving from state preparing -> state finished
>Thread-136::DEBUG::2012-12-09 12:09:52,409::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-136::DEBUG::2012-12-09 12:09:52,409::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-136::DEBUG::2012-12-09 12:09:52,410::task::978::TaskManager.Task::(_decref) Task=`f46afdfd-0b8c-4429-9b5b-651a4080ddff`::ref 0 aborting False
>Thread-138::DEBUG::2012-12-09 12:09:53,234::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-138::DEBUG::2012-12-09 12:09:53,235::task::588::TaskManager.Task::(_updateState) Task=`207112a5-2b29-4f93-9d9c-6c4dc44f752c`::moving from state init -> state preparing
>Thread-138::INFO::2012-12-09 12:09:53,235::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-138::DEBUG::2012-12-09 12:09:53,236::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-138::DEBUG::2012-12-09 12:09:53,236::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-138::INFO::2012-12-09 12:09:53,236::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-138::DEBUG::2012-12-09 12:09:53,237::task::1172::TaskManager.Task::(prepare) Task=`207112a5-2b29-4f93-9d9c-6c4dc44f752c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-138::DEBUG::2012-12-09 12:09:53,237::task::588::TaskManager.Task::(_updateState) Task=`207112a5-2b29-4f93-9d9c-6c4dc44f752c`::moving from state preparing -> state finished
>Thread-138::DEBUG::2012-12-09 12:09:53,238::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-138::DEBUG::2012-12-09 12:09:53,238::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-138::DEBUG::2012-12-09 12:09:53,238::task::978::TaskManager.Task::(_decref) Task=`207112a5-2b29-4f93-9d9c-6c4dc44f752c`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:09:53,541::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:09:53,542::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:09:53,662::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:09:53,664::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:09:53,664::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:09:53,665::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-139::DEBUG::2012-12-09 12:09:54,271::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-139::DEBUG::2012-12-09 12:09:54,272::task::588::TaskManager.Task::(_updateState) Task=`aacb8cdf-7006-49f3-bc27-b58e6df9168a`::moving from state init -> state preparing
>Thread-139::INFO::2012-12-09 12:09:54,273::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-139::DEBUG::2012-12-09 12:09:54,273::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-139::DEBUG::2012-12-09 12:09:54,274::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-139::INFO::2012-12-09 12:09:54,274::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-139::DEBUG::2012-12-09 12:09:54,275::task::1172::TaskManager.Task::(prepare) Task=`aacb8cdf-7006-49f3-bc27-b58e6df9168a`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-139::DEBUG::2012-12-09 12:09:54,275::task::588::TaskManager.Task::(_updateState) Task=`aacb8cdf-7006-49f3-bc27-b58e6df9168a`::moving from state preparing -> state finished
>Thread-139::DEBUG::2012-12-09 12:09:54,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-139::DEBUG::2012-12-09 12:09:54,276::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-139::DEBUG::2012-12-09 12:09:54,276::task::978::TaskManager.Task::(_decref) Task=`aacb8cdf-7006-49f3-bc27-b58e6df9168a`::ref 0 aborting False
>Thread-141::DEBUG::2012-12-09 12:09:55,304::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-141::DEBUG::2012-12-09 12:09:55,305::task::588::TaskManager.Task::(_updateState) Task=`7ffe08ac-b52d-4da9-b232-a209be405974`::moving from state init -> state preparing
>Thread-141::INFO::2012-12-09 12:09:55,306::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-141::DEBUG::2012-12-09 12:09:55,306::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-141::DEBUG::2012-12-09 12:09:55,307::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-141::INFO::2012-12-09 12:09:55,307::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-141::DEBUG::2012-12-09 12:09:55,307::task::1172::TaskManager.Task::(prepare) Task=`7ffe08ac-b52d-4da9-b232-a209be405974`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-141::DEBUG::2012-12-09 12:09:55,308::task::588::TaskManager.Task::(_updateState) Task=`7ffe08ac-b52d-4da9-b232-a209be405974`::moving from state preparing -> state finished
>Thread-141::DEBUG::2012-12-09 12:09:55,308::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-141::DEBUG::2012-12-09 12:09:55,308::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-141::DEBUG::2012-12-09 12:09:55,309::task::978::TaskManager.Task::(_decref) Task=`7ffe08ac-b52d-4da9-b232-a209be405974`::ref 0 aborting False
>Thread-142::DEBUG::2012-12-09 12:09:56,341::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-142::DEBUG::2012-12-09 12:09:56,342::task::588::TaskManager.Task::(_updateState) Task=`fc822893-1031-4944-9ca8-3d96ad71fb69`::moving from state init -> state preparing
>Thread-142::INFO::2012-12-09 12:09:56,342::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-142::DEBUG::2012-12-09 12:09:56,342::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-142::DEBUG::2012-12-09 12:09:56,342::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-142::INFO::2012-12-09 12:09:56,342::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-142::DEBUG::2012-12-09 12:09:56,343::task::1172::TaskManager.Task::(prepare) Task=`fc822893-1031-4944-9ca8-3d96ad71fb69`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-142::DEBUG::2012-12-09 12:09:56,343::task::588::TaskManager.Task::(_updateState) Task=`fc822893-1031-4944-9ca8-3d96ad71fb69`::moving from state preparing -> state finished
>Thread-142::DEBUG::2012-12-09 12:09:56,343::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-142::DEBUG::2012-12-09 12:09:56,343::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-142::DEBUG::2012-12-09 12:09:56,343::task::978::TaskManager.Task::(_decref) Task=`fc822893-1031-4944-9ca8-3d96ad71fb69`::ref 0 aborting False
>Thread-144::DEBUG::2012-12-09 12:09:57,373::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-144::DEBUG::2012-12-09 12:09:57,374::task::588::TaskManager.Task::(_updateState) Task=`6a41c08e-029b-43d7-9f06-e2f4d14acc12`::moving from state init -> state preparing
>Thread-144::INFO::2012-12-09 12:09:57,375::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-144::DEBUG::2012-12-09 12:09:57,375::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-144::DEBUG::2012-12-09 12:09:57,375::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-144::INFO::2012-12-09 12:09:57,376::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-144::DEBUG::2012-12-09 12:09:57,376::task::1172::TaskManager.Task::(prepare) Task=`6a41c08e-029b-43d7-9f06-e2f4d14acc12`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-144::DEBUG::2012-12-09 12:09:57,377::task::588::TaskManager.Task::(_updateState) Task=`6a41c08e-029b-43d7-9f06-e2f4d14acc12`::moving from state preparing -> state finished
>Thread-144::DEBUG::2012-12-09 12:09:57,377::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-144::DEBUG::2012-12-09 12:09:57,377::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-144::DEBUG::2012-12-09 12:09:57,378::task::978::TaskManager.Task::(_decref) Task=`6a41c08e-029b-43d7-9f06-e2f4d14acc12`::ref 0 aborting False
>Thread-145::DEBUG::2012-12-09 12:09:58,406::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-145::DEBUG::2012-12-09 12:09:58,407::task::588::TaskManager.Task::(_updateState) Task=`23188491-fbca-4050-8566-2e71b4955575`::moving from state init -> state preparing
>Thread-145::INFO::2012-12-09 12:09:58,407::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-145::DEBUG::2012-12-09 12:09:58,407::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-145::DEBUG::2012-12-09 12:09:58,408::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-145::INFO::2012-12-09 12:09:58,408::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-145::DEBUG::2012-12-09 12:09:58,409::task::1172::TaskManager.Task::(prepare) Task=`23188491-fbca-4050-8566-2e71b4955575`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-145::DEBUG::2012-12-09 12:09:58,409::task::588::TaskManager.Task::(_updateState) Task=`23188491-fbca-4050-8566-2e71b4955575`::moving from state preparing -> state finished
>Thread-145::DEBUG::2012-12-09 12:09:58,409::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-145::DEBUG::2012-12-09 12:09:58,410::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-145::DEBUG::2012-12-09 12:09:58,410::task::978::TaskManager.Task::(_decref) Task=`23188491-fbca-4050-8566-2e71b4955575`::ref 0 aborting False
>Thread-147::DEBUG::2012-12-09 12:09:59,444::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-147::DEBUG::2012-12-09 12:09:59,444::task::588::TaskManager.Task::(_updateState) Task=`972b47ad-32fa-4782-8ed3-e1c0a0d95b39`::moving from state init -> state preparing
>Thread-147::INFO::2012-12-09 12:09:59,444::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-147::DEBUG::2012-12-09 12:09:59,445::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-147::DEBUG::2012-12-09 12:09:59,445::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-147::INFO::2012-12-09 12:09:59,445::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-147::DEBUG::2012-12-09 12:09:59,445::task::1172::TaskManager.Task::(prepare) Task=`972b47ad-32fa-4782-8ed3-e1c0a0d95b39`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-147::DEBUG::2012-12-09 12:09:59,446::task::588::TaskManager.Task::(_updateState) Task=`972b47ad-32fa-4782-8ed3-e1c0a0d95b39`::moving from state preparing -> state finished
>Thread-147::DEBUG::2012-12-09 12:09:59,446::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-147::DEBUG::2012-12-09 12:09:59,446::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-147::DEBUG::2012-12-09 12:09:59,446::task::978::TaskManager.Task::(_decref) Task=`972b47ad-32fa-4782-8ed3-e1c0a0d95b39`::ref 0 aborting False
>Thread-148::DEBUG::2012-12-09 12:10:00,476::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-148::DEBUG::2012-12-09 12:10:00,477::task::588::TaskManager.Task::(_updateState) Task=`1942bd25-dbf2-44da-bb7e-1675bc18f737`::moving from state init -> state preparing
>Thread-148::INFO::2012-12-09 12:10:00,477::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-148::DEBUG::2012-12-09 12:10:00,478::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-148::DEBUG::2012-12-09 12:10:00,478::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-148::INFO::2012-12-09 12:10:00,478::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-148::DEBUG::2012-12-09 12:10:00,479::task::1172::TaskManager.Task::(prepare) Task=`1942bd25-dbf2-44da-bb7e-1675bc18f737`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-148::DEBUG::2012-12-09 12:10:00,479::task::588::TaskManager.Task::(_updateState) Task=`1942bd25-dbf2-44da-bb7e-1675bc18f737`::moving from state preparing -> state finished
>Thread-148::DEBUG::2012-12-09 12:10:00,479::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-148::DEBUG::2012-12-09 12:10:00,479::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-148::DEBUG::2012-12-09 12:10:00,479::task::978::TaskManager.Task::(_decref) Task=`1942bd25-dbf2-44da-bb7e-1675bc18f737`::ref 0 aborting False
>Thread-150::DEBUG::2012-12-09 12:10:01,507::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-150::DEBUG::2012-12-09 12:10:01,513::task::588::TaskManager.Task::(_updateState) Task=`014d1d5a-eaa5-427c-9841-90b64eff2f58`::moving from state init -> state preparing
>Thread-150::INFO::2012-12-09 12:10:01,514::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-150::DEBUG::2012-12-09 12:10:01,514::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-150::DEBUG::2012-12-09 12:10:01,515::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-150::INFO::2012-12-09 12:10:01,515::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-150::DEBUG::2012-12-09 12:10:01,515::task::1172::TaskManager.Task::(prepare) Task=`014d1d5a-eaa5-427c-9841-90b64eff2f58`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-150::DEBUG::2012-12-09 12:10:01,516::task::588::TaskManager.Task::(_updateState) Task=`014d1d5a-eaa5-427c-9841-90b64eff2f58`::moving from state preparing -> state finished
>Thread-150::DEBUG::2012-12-09 12:10:01,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-150::DEBUG::2012-12-09 12:10:01,516::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-150::DEBUG::2012-12-09 12:10:01,516::task::978::TaskManager.Task::(_decref) Task=`014d1d5a-eaa5-427c-9841-90b64eff2f58`::ref 0 aborting False
>Thread-151::DEBUG::2012-12-09 12:10:02,552::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-151::DEBUG::2012-12-09 12:10:02,553::task::588::TaskManager.Task::(_updateState) Task=`1387c329-089c-4b65-abee-7ddb608c7622`::moving from state init -> state preparing
>Thread-151::INFO::2012-12-09 12:10:02,554::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-151::DEBUG::2012-12-09 12:10:02,554::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-151::DEBUG::2012-12-09 12:10:02,554::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-151::INFO::2012-12-09 12:10:02,555::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-151::DEBUG::2012-12-09 12:10:02,555::task::1172::TaskManager.Task::(prepare) Task=`1387c329-089c-4b65-abee-7ddb608c7622`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-151::DEBUG::2012-12-09 12:10:02,556::task::588::TaskManager.Task::(_updateState) Task=`1387c329-089c-4b65-abee-7ddb608c7622`::moving from state preparing -> state finished
>Thread-151::DEBUG::2012-12-09 12:10:02,556::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-151::DEBUG::2012-12-09 12:10:02,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-151::DEBUG::2012-12-09 12:10:02,557::task::978::TaskManager.Task::(_decref) Task=`1387c329-089c-4b65-abee-7ddb608c7622`::ref 0 aborting False
>Thread-152::DEBUG::2012-12-09 12:10:02,668::task::588::TaskManager.Task::(_updateState) Task=`4a3b8cfb-77db-4d0d-a6d4-dc53be0d0a07`::moving from state init -> state preparing
>Thread-152::INFO::2012-12-09 12:10:02,669::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-152::INFO::2012-12-09 12:10:02,669::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '59.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.0', 'code': 358, 'valid': False}}
>Thread-152::DEBUG::2012-12-09 12:10:02,669::task::1172::TaskManager.Task::(prepare) Task=`4a3b8cfb-77db-4d0d-a6d4-dc53be0d0a07`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '59.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.0', 'code': 358, 'valid': False}}
>Thread-152::DEBUG::2012-12-09 12:10:02,669::task::588::TaskManager.Task::(_updateState) Task=`4a3b8cfb-77db-4d0d-a6d4-dc53be0d0a07`::moving from state preparing -> state finished
>Thread-152::DEBUG::2012-12-09 12:10:02,670::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-152::DEBUG::2012-12-09 12:10:02,670::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-152::DEBUG::2012-12-09 12:10:02,671::task::978::TaskManager.Task::(_decref) Task=`4a3b8cfb-77db-4d0d-a6d4-dc53be0d0a07`::ref 0 aborting False
>Thread-154::DEBUG::2012-12-09 12:10:03,600::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-154::DEBUG::2012-12-09 12:10:03,601::task::588::TaskManager.Task::(_updateState) Task=`39e0ad4a-b5a0-4bf3-9e63-2c555a96b6fe`::moving from state init -> state preparing
>Thread-154::INFO::2012-12-09 12:10:03,602::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-154::DEBUG::2012-12-09 12:10:03,603::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-154::DEBUG::2012-12-09 12:10:03,603::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-154::INFO::2012-12-09 12:10:03,604::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-154::DEBUG::2012-12-09 12:10:03,605::task::1172::TaskManager.Task::(prepare) Task=`39e0ad4a-b5a0-4bf3-9e63-2c555a96b6fe`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-154::DEBUG::2012-12-09 12:10:03,605::task::588::TaskManager.Task::(_updateState) Task=`39e0ad4a-b5a0-4bf3-9e63-2c555a96b6fe`::moving from state preparing -> state finished
>Thread-154::DEBUG::2012-12-09 12:10:03,606::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-154::DEBUG::2012-12-09 12:10:03,606::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-154::DEBUG::2012-12-09 12:10:03,607::task::978::TaskManager.Task::(_decref) Task=`39e0ad4a-b5a0-4bf3-9e63-2c555a96b6fe`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:03,667::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:03,667::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:10:03,783::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:03,785::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:03,786::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:03,787::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-155::DEBUG::2012-12-09 12:10:04,703::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-155::DEBUG::2012-12-09 12:10:04,703::task::588::TaskManager.Task::(_updateState) Task=`fd947a76-e8a8-4803-8255-cf05756d4d42`::moving from state init -> state preparing
>Thread-155::INFO::2012-12-09 12:10:04,704::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-155::DEBUG::2012-12-09 12:10:04,704::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-155::DEBUG::2012-12-09 12:10:04,704::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-155::INFO::2012-12-09 12:10:04,704::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-155::DEBUG::2012-12-09 12:10:04,705::task::1172::TaskManager.Task::(prepare) Task=`fd947a76-e8a8-4803-8255-cf05756d4d42`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-155::DEBUG::2012-12-09 12:10:04,705::task::588::TaskManager.Task::(_updateState) Task=`fd947a76-e8a8-4803-8255-cf05756d4d42`::moving from state preparing -> state finished
>Thread-155::DEBUG::2012-12-09 12:10:04,705::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-155::DEBUG::2012-12-09 12:10:04,705::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-155::DEBUG::2012-12-09 12:10:04,705::task::978::TaskManager.Task::(_decref) Task=`fd947a76-e8a8-4803-8255-cf05756d4d42`::ref 0 aborting False
>Thread-157::DEBUG::2012-12-09 12:10:05,735::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-157::DEBUG::2012-12-09 12:10:05,736::task::588::TaskManager.Task::(_updateState) Task=`17763849-d748-4047-952e-4b521ab99055`::moving from state init -> state preparing
>Thread-157::INFO::2012-12-09 12:10:05,736::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-157::DEBUG::2012-12-09 12:10:05,737::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-157::DEBUG::2012-12-09 12:10:05,737::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-157::INFO::2012-12-09 12:10:05,737::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-157::DEBUG::2012-12-09 12:10:05,738::task::1172::TaskManager.Task::(prepare) Task=`17763849-d748-4047-952e-4b521ab99055`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-157::DEBUG::2012-12-09 12:10:05,738::task::588::TaskManager.Task::(_updateState) Task=`17763849-d748-4047-952e-4b521ab99055`::moving from state preparing -> state finished
>Thread-157::DEBUG::2012-12-09 12:10:05,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-157::DEBUG::2012-12-09 12:10:05,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-157::DEBUG::2012-12-09 12:10:05,739::task::978::TaskManager.Task::(_decref) Task=`17763849-d748-4047-952e-4b521ab99055`::ref 0 aborting False
>Thread-158::DEBUG::2012-12-09 12:10:06,766::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-158::DEBUG::2012-12-09 12:10:06,767::task::588::TaskManager.Task::(_updateState) Task=`ae2cc2bf-23be-45e2-9b5d-5e8762bf9d59`::moving from state init -> state preparing
>Thread-158::INFO::2012-12-09 12:10:06,767::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-158::DEBUG::2012-12-09 12:10:06,768::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-158::DEBUG::2012-12-09 12:10:06,768::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-158::INFO::2012-12-09 12:10:06,768::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-158::DEBUG::2012-12-09 12:10:06,769::task::1172::TaskManager.Task::(prepare) Task=`ae2cc2bf-23be-45e2-9b5d-5e8762bf9d59`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-158::DEBUG::2012-12-09 12:10:06,769::task::588::TaskManager.Task::(_updateState) Task=`ae2cc2bf-23be-45e2-9b5d-5e8762bf9d59`::moving from state preparing -> state finished
>Thread-158::DEBUG::2012-12-09 12:10:06,769::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-158::DEBUG::2012-12-09 12:10:06,770::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-158::DEBUG::2012-12-09 12:10:06,770::task::978::TaskManager.Task::(_decref) Task=`ae2cc2bf-23be-45e2-9b5d-5e8762bf9d59`::ref 0 aborting False
>Thread-160::DEBUG::2012-12-09 12:10:07,806::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-160::DEBUG::2012-12-09 12:10:07,806::task::588::TaskManager.Task::(_updateState) Task=`e65a6d62-df9f-426f-8ee5-d56d734cd6ee`::moving from state init -> state preparing
>Thread-160::INFO::2012-12-09 12:10:07,806::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-160::DEBUG::2012-12-09 12:10:07,807::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-160::DEBUG::2012-12-09 12:10:07,807::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-160::INFO::2012-12-09 12:10:07,807::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-160::DEBUG::2012-12-09 12:10:07,807::task::1172::TaskManager.Task::(prepare) Task=`e65a6d62-df9f-426f-8ee5-d56d734cd6ee`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-160::DEBUG::2012-12-09 12:10:07,807::task::588::TaskManager.Task::(_updateState) Task=`e65a6d62-df9f-426f-8ee5-d56d734cd6ee`::moving from state preparing -> state finished
>Thread-160::DEBUG::2012-12-09 12:10:07,807::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-160::DEBUG::2012-12-09 12:10:07,808::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-160::DEBUG::2012-12-09 12:10:07,808::task::978::TaskManager.Task::(_decref) Task=`e65a6d62-df9f-426f-8ee5-d56d734cd6ee`::ref 0 aborting False
>Thread-161::DEBUG::2012-12-09 12:10:08,851::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-161::DEBUG::2012-12-09 12:10:08,852::task::588::TaskManager.Task::(_updateState) Task=`f26af82a-036e-4107-af60-ab235be13788`::moving from state init -> state preparing
>Thread-161::INFO::2012-12-09 12:10:08,853::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-161::DEBUG::2012-12-09 12:10:08,853::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-161::DEBUG::2012-12-09 12:10:08,853::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-161::INFO::2012-12-09 12:10:08,854::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-161::DEBUG::2012-12-09 12:10:08,854::task::1172::TaskManager.Task::(prepare) Task=`f26af82a-036e-4107-af60-ab235be13788`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-161::DEBUG::2012-12-09 12:10:08,855::task::588::TaskManager.Task::(_updateState) Task=`f26af82a-036e-4107-af60-ab235be13788`::moving from state preparing -> state finished
>Thread-161::DEBUG::2012-12-09 12:10:08,855::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-161::DEBUG::2012-12-09 12:10:08,855::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-161::DEBUG::2012-12-09 12:10:08,856::task::978::TaskManager.Task::(_decref) Task=`f26af82a-036e-4107-af60-ab235be13788`::ref 0 aborting False
>Thread-163::DEBUG::2012-12-09 12:10:09,888::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-163::DEBUG::2012-12-09 12:10:09,889::task::588::TaskManager.Task::(_updateState) Task=`0d09234a-8d20-4250-9bf8-1151f35b6d74`::moving from state init -> state preparing
>Thread-163::INFO::2012-12-09 12:10:09,889::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-163::DEBUG::2012-12-09 12:10:09,890::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-163::DEBUG::2012-12-09 12:10:09,890::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-163::INFO::2012-12-09 12:10:09,891::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-163::DEBUG::2012-12-09 12:10:09,891::task::1172::TaskManager.Task::(prepare) Task=`0d09234a-8d20-4250-9bf8-1151f35b6d74`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-163::DEBUG::2012-12-09 12:10:09,892::task::588::TaskManager.Task::(_updateState) Task=`0d09234a-8d20-4250-9bf8-1151f35b6d74`::moving from state preparing -> state finished
>Thread-163::DEBUG::2012-12-09 12:10:09,892::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-163::DEBUG::2012-12-09 12:10:09,892::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-163::DEBUG::2012-12-09 12:10:09,893::task::978::TaskManager.Task::(_decref) Task=`0d09234a-8d20-4250-9bf8-1151f35b6d74`::ref 0 aborting False
>Thread-164::DEBUG::2012-12-09 12:10:10,923::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-164::DEBUG::2012-12-09 12:10:10,924::task::588::TaskManager.Task::(_updateState) Task=`cc808663-fe80-4930-9086-83bca0e29986`::moving from state init -> state preparing
>Thread-164::INFO::2012-12-09 12:10:10,924::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-164::DEBUG::2012-12-09 12:10:10,925::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-164::DEBUG::2012-12-09 12:10:10,925::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-164::INFO::2012-12-09 12:10:10,926::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-164::DEBUG::2012-12-09 12:10:10,926::task::1172::TaskManager.Task::(prepare) Task=`cc808663-fe80-4930-9086-83bca0e29986`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-164::DEBUG::2012-12-09 12:10:10,926::task::588::TaskManager.Task::(_updateState) Task=`cc808663-fe80-4930-9086-83bca0e29986`::moving from state preparing -> state finished
>Thread-164::DEBUG::2012-12-09 12:10:10,927::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-164::DEBUG::2012-12-09 12:10:10,927::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-164::DEBUG::2012-12-09 12:10:10,928::task::978::TaskManager.Task::(_decref) Task=`cc808663-fe80-4930-9086-83bca0e29986`::ref 0 aborting False
>Thread-166::DEBUG::2012-12-09 12:10:11,962::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-166::DEBUG::2012-12-09 12:10:11,962::task::588::TaskManager.Task::(_updateState) Task=`6d6dc54e-a3a8-40bb-afc9-dc0d0aa5ef83`::moving from state init -> state preparing
>Thread-166::INFO::2012-12-09 12:10:11,963::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-166::DEBUG::2012-12-09 12:10:11,963::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-166::DEBUG::2012-12-09 12:10:11,963::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-166::INFO::2012-12-09 12:10:11,963::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-166::DEBUG::2012-12-09 12:10:11,963::task::1172::TaskManager.Task::(prepare) Task=`6d6dc54e-a3a8-40bb-afc9-dc0d0aa5ef83`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-166::DEBUG::2012-12-09 12:10:11,964::task::588::TaskManager.Task::(_updateState) Task=`6d6dc54e-a3a8-40bb-afc9-dc0d0aa5ef83`::moving from state preparing -> state finished
>Thread-166::DEBUG::2012-12-09 12:10:11,964::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-166::DEBUG::2012-12-09 12:10:11,965::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-166::DEBUG::2012-12-09 12:10:11,965::task::978::TaskManager.Task::(_decref) Task=`6d6dc54e-a3a8-40bb-afc9-dc0d0aa5ef83`::ref 0 aborting False
>Thread-167::DEBUG::2012-12-09 12:10:12,995::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-167::DEBUG::2012-12-09 12:10:12,996::task::588::TaskManager.Task::(_updateState) Task=`d4659bec-35f8-46ea-8e82-c46f95f6c962`::moving from state init -> state preparing
>Thread-167::INFO::2012-12-09 12:10:12,997::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-167::DEBUG::2012-12-09 12:10:12,997::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-167::DEBUG::2012-12-09 12:10:12,997::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-167::INFO::2012-12-09 12:10:12,997::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-167::DEBUG::2012-12-09 12:10:12,997::task::1172::TaskManager.Task::(prepare) Task=`d4659bec-35f8-46ea-8e82-c46f95f6c962`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-167::DEBUG::2012-12-09 12:10:12,998::task::588::TaskManager.Task::(_updateState) Task=`d4659bec-35f8-46ea-8e82-c46f95f6c962`::moving from state preparing -> state finished
>Thread-167::DEBUG::2012-12-09 12:10:12,998::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-167::DEBUG::2012-12-09 12:10:12,998::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-167::DEBUG::2012-12-09 12:10:12,998::task::978::TaskManager.Task::(_decref) Task=`d4659bec-35f8-46ea-8e82-c46f95f6c962`::ref 0 aborting False
>Thread-168::DEBUG::2012-12-09 12:10:13,016::task::588::TaskManager.Task::(_updateState) Task=`d9f1e92f-1519-4d2b-bf8c-0cbf1a1476e7`::moving from state init -> state preparing
>Thread-168::INFO::2012-12-09 12:10:13,017::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-168::INFO::2012-12-09 12:10:13,017::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '69.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-168::DEBUG::2012-12-09 12:10:13,018::task::1172::TaskManager.Task::(prepare) Task=`d9f1e92f-1519-4d2b-bf8c-0cbf1a1476e7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '69.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-168::DEBUG::2012-12-09 12:10:13,018::task::588::TaskManager.Task::(_updateState) Task=`d9f1e92f-1519-4d2b-bf8c-0cbf1a1476e7`::moving from state preparing -> state finished
>Thread-168::DEBUG::2012-12-09 12:10:13,019::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-168::DEBUG::2012-12-09 12:10:13,019::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-168::DEBUG::2012-12-09 12:10:13,019::task::978::TaskManager.Task::(_decref) Task=`d9f1e92f-1519-4d2b-bf8c-0cbf1a1476e7`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:13,788::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:13,789::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:10:13,900::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:13,901::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:13,902::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:13,902::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-170::DEBUG::2012-12-09 12:10:14,032::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-170::DEBUG::2012-12-09 12:10:14,033::task::588::TaskManager.Task::(_updateState) Task=`d65c3d62-ef9f-4634-8669-ae45efa26ec5`::moving from state init -> state preparing
>Thread-170::INFO::2012-12-09 12:10:14,034::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-170::DEBUG::2012-12-09 12:10:14,034::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-170::DEBUG::2012-12-09 12:10:14,034::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-170::INFO::2012-12-09 12:10:14,035::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-170::DEBUG::2012-12-09 12:10:14,035::task::1172::TaskManager.Task::(prepare) Task=`d65c3d62-ef9f-4634-8669-ae45efa26ec5`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-170::DEBUG::2012-12-09 12:10:14,036::task::588::TaskManager.Task::(_updateState) Task=`d65c3d62-ef9f-4634-8669-ae45efa26ec5`::moving from state preparing -> state finished
>Thread-170::DEBUG::2012-12-09 12:10:14,036::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-170::DEBUG::2012-12-09 12:10:14,036::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-170::DEBUG::2012-12-09 12:10:14,037::task::978::TaskManager.Task::(_decref) Task=`d65c3d62-ef9f-4634-8669-ae45efa26ec5`::ref 0 aborting False
>Thread-171::DEBUG::2012-12-09 12:10:15,071::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-171::DEBUG::2012-12-09 12:10:15,072::task::588::TaskManager.Task::(_updateState) Task=`3430985e-b62c-4ad4-a538-358ace0c93ab`::moving from state init -> state preparing
>Thread-171::INFO::2012-12-09 12:10:15,072::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-171::DEBUG::2012-12-09 12:10:15,072::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-171::DEBUG::2012-12-09 12:10:15,073::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-171::INFO::2012-12-09 12:10:15,073::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-171::DEBUG::2012-12-09 12:10:15,073::task::1172::TaskManager.Task::(prepare) Task=`3430985e-b62c-4ad4-a538-358ace0c93ab`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-171::DEBUG::2012-12-09 12:10:15,074::task::588::TaskManager.Task::(_updateState) Task=`3430985e-b62c-4ad4-a538-358ace0c93ab`::moving from state preparing -> state finished
>Thread-171::DEBUG::2012-12-09 12:10:15,074::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-171::DEBUG::2012-12-09 12:10:15,075::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-171::DEBUG::2012-12-09 12:10:15,075::task::978::TaskManager.Task::(_decref) Task=`3430985e-b62c-4ad4-a538-358ace0c93ab`::ref 0 aborting False
>Thread-173::DEBUG::2012-12-09 12:10:16,103::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-173::DEBUG::2012-12-09 12:10:16,103::task::588::TaskManager.Task::(_updateState) Task=`7983f488-0516-4610-b64a-157c1f6d943b`::moving from state init -> state preparing
>Thread-173::INFO::2012-12-09 12:10:16,104::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-173::DEBUG::2012-12-09 12:10:16,104::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-173::DEBUG::2012-12-09 12:10:16,104::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-173::INFO::2012-12-09 12:10:16,104::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-173::DEBUG::2012-12-09 12:10:16,104::task::1172::TaskManager.Task::(prepare) Task=`7983f488-0516-4610-b64a-157c1f6d943b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-173::DEBUG::2012-12-09 12:10:16,105::task::588::TaskManager.Task::(_updateState) Task=`7983f488-0516-4610-b64a-157c1f6d943b`::moving from state preparing -> state finished
>Thread-173::DEBUG::2012-12-09 12:10:16,105::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-173::DEBUG::2012-12-09 12:10:16,105::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-173::DEBUG::2012-12-09 12:10:16,105::task::978::TaskManager.Task::(_decref) Task=`7983f488-0516-4610-b64a-157c1f6d943b`::ref 0 aborting False
>Thread-174::DEBUG::2012-12-09 12:10:17,127::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-174::DEBUG::2012-12-09 12:10:17,128::task::588::TaskManager.Task::(_updateState) Task=`5032e922-b1fe-4fd1-ba80-a4af711516a0`::moving from state init -> state preparing
>Thread-174::INFO::2012-12-09 12:10:17,129::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-174::DEBUG::2012-12-09 12:10:17,129::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-174::DEBUG::2012-12-09 12:10:17,129::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-174::INFO::2012-12-09 12:10:17,130::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-174::DEBUG::2012-12-09 12:10:17,130::task::1172::TaskManager.Task::(prepare) Task=`5032e922-b1fe-4fd1-ba80-a4af711516a0`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-174::DEBUG::2012-12-09 12:10:17,130::task::588::TaskManager.Task::(_updateState) Task=`5032e922-b1fe-4fd1-ba80-a4af711516a0`::moving from state preparing -> state finished
>Thread-174::DEBUG::2012-12-09 12:10:17,130::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-174::DEBUG::2012-12-09 12:10:17,130::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-174::DEBUG::2012-12-09 12:10:17,130::task::978::TaskManager.Task::(_decref) Task=`5032e922-b1fe-4fd1-ba80-a4af711516a0`::ref 0 aborting False
>Thread-176::DEBUG::2012-12-09 12:10:18,162::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-176::DEBUG::2012-12-09 12:10:18,163::task::588::TaskManager.Task::(_updateState) Task=`20e3f2d7-66c1-424e-add7-c0164f6ce876`::moving from state init -> state preparing
>Thread-176::INFO::2012-12-09 12:10:18,163::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-176::DEBUG::2012-12-09 12:10:18,164::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-176::DEBUG::2012-12-09 12:10:18,164::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-176::INFO::2012-12-09 12:10:18,165::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-176::DEBUG::2012-12-09 12:10:18,165::task::1172::TaskManager.Task::(prepare) Task=`20e3f2d7-66c1-424e-add7-c0164f6ce876`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-176::DEBUG::2012-12-09 12:10:18,165::task::588::TaskManager.Task::(_updateState) Task=`20e3f2d7-66c1-424e-add7-c0164f6ce876`::moving from state preparing -> state finished
>Thread-176::DEBUG::2012-12-09 12:10:18,166::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-176::DEBUG::2012-12-09 12:10:18,166::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-176::DEBUG::2012-12-09 12:10:18,167::task::978::TaskManager.Task::(_decref) Task=`20e3f2d7-66c1-424e-add7-c0164f6ce876`::ref 0 aborting False
>Thread-177::DEBUG::2012-12-09 12:10:19,195::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-177::DEBUG::2012-12-09 12:10:19,195::task::588::TaskManager.Task::(_updateState) Task=`41416446-11a3-4789-b599-1ae9cec19558`::moving from state init -> state preparing
>Thread-177::INFO::2012-12-09 12:10:19,196::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-177::DEBUG::2012-12-09 12:10:19,196::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-177::DEBUG::2012-12-09 12:10:19,196::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-177::INFO::2012-12-09 12:10:19,197::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-177::DEBUG::2012-12-09 12:10:19,197::task::1172::TaskManager.Task::(prepare) Task=`41416446-11a3-4789-b599-1ae9cec19558`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-177::DEBUG::2012-12-09 12:10:19,197::task::588::TaskManager.Task::(_updateState) Task=`41416446-11a3-4789-b599-1ae9cec19558`::moving from state preparing -> state finished
>Thread-177::DEBUG::2012-12-09 12:10:19,197::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-177::DEBUG::2012-12-09 12:10:19,198::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-177::DEBUG::2012-12-09 12:10:19,198::task::978::TaskManager.Task::(_decref) Task=`41416446-11a3-4789-b599-1ae9cec19558`::ref 0 aborting False
>Thread-179::DEBUG::2012-12-09 12:10:20,219::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-179::DEBUG::2012-12-09 12:10:20,219::task::588::TaskManager.Task::(_updateState) Task=`8bf6cd93-4c0c-4bf9-a93d-acf20fd56564`::moving from state init -> state preparing
>Thread-179::INFO::2012-12-09 12:10:20,220::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-179::DEBUG::2012-12-09 12:10:20,220::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-179::DEBUG::2012-12-09 12:10:20,220::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-179::INFO::2012-12-09 12:10:20,220::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-179::DEBUG::2012-12-09 12:10:20,220::task::1172::TaskManager.Task::(prepare) Task=`8bf6cd93-4c0c-4bf9-a93d-acf20fd56564`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-179::DEBUG::2012-12-09 12:10:20,220::task::588::TaskManager.Task::(_updateState) Task=`8bf6cd93-4c0c-4bf9-a93d-acf20fd56564`::moving from state preparing -> state finished
>Thread-179::DEBUG::2012-12-09 12:10:20,221::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-179::DEBUG::2012-12-09 12:10:20,221::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-179::DEBUG::2012-12-09 12:10:20,221::task::978::TaskManager.Task::(_decref) Task=`8bf6cd93-4c0c-4bf9-a93d-acf20fd56564`::ref 0 aborting False
>Thread-180::DEBUG::2012-12-09 12:10:21,250::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-180::DEBUG::2012-12-09 12:10:21,251::task::588::TaskManager.Task::(_updateState) Task=`eb14b213-835c-4cad-a443-3cb6f18ee6b0`::moving from state init -> state preparing
>Thread-180::INFO::2012-12-09 12:10:21,252::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-180::DEBUG::2012-12-09 12:10:21,252::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-180::DEBUG::2012-12-09 12:10:21,252::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-180::INFO::2012-12-09 12:10:21,253::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-180::DEBUG::2012-12-09 12:10:21,253::task::1172::TaskManager.Task::(prepare) Task=`eb14b213-835c-4cad-a443-3cb6f18ee6b0`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-180::DEBUG::2012-12-09 12:10:21,254::task::588::TaskManager.Task::(_updateState) Task=`eb14b213-835c-4cad-a443-3cb6f18ee6b0`::moving from state preparing -> state finished
>Thread-180::DEBUG::2012-12-09 12:10:21,254::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-180::DEBUG::2012-12-09 12:10:21,254::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-180::DEBUG::2012-12-09 12:10:21,254::task::978::TaskManager.Task::(_decref) Task=`eb14b213-835c-4cad-a443-3cb6f18ee6b0`::ref 0 aborting False
>Thread-182::DEBUG::2012-12-09 12:10:22,285::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-182::DEBUG::2012-12-09 12:10:22,285::task::588::TaskManager.Task::(_updateState) Task=`f9d6fded-18c1-4bc2-a928-c25c1c6105ec`::moving from state init -> state preparing
>Thread-182::INFO::2012-12-09 12:10:22,286::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-182::DEBUG::2012-12-09 12:10:22,286::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-182::DEBUG::2012-12-09 12:10:22,287::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-182::INFO::2012-12-09 12:10:22,287::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-182::DEBUG::2012-12-09 12:10:22,288::task::1172::TaskManager.Task::(prepare) Task=`f9d6fded-18c1-4bc2-a928-c25c1c6105ec`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-182::DEBUG::2012-12-09 12:10:22,288::task::588::TaskManager.Task::(_updateState) Task=`f9d6fded-18c1-4bc2-a928-c25c1c6105ec`::moving from state preparing -> state finished
>Thread-182::DEBUG::2012-12-09 12:10:22,288::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-182::DEBUG::2012-12-09 12:10:22,289::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-182::DEBUG::2012-12-09 12:10:22,289::task::978::TaskManager.Task::(_decref) Task=`f9d6fded-18c1-4bc2-a928-c25c1c6105ec`::ref 0 aborting False
>Thread-183::DEBUG::2012-12-09 12:10:23,274::task::588::TaskManager.Task::(_updateState) Task=`d4f76815-a3e9-4971-b4c9-f6a9c29dfb75`::moving from state init -> state preparing
>Thread-183::INFO::2012-12-09 12:10:23,274::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-183::INFO::2012-12-09 12:10:23,275::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '80.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-183::DEBUG::2012-12-09 12:10:23,275::task::1172::TaskManager.Task::(prepare) Task=`d4f76815-a3e9-4971-b4c9-f6a9c29dfb75`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '80.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-183::DEBUG::2012-12-09 12:10:23,275::task::588::TaskManager.Task::(_updateState) Task=`d4f76815-a3e9-4971-b4c9-f6a9c29dfb75`::moving from state preparing -> state finished
>Thread-183::DEBUG::2012-12-09 12:10:23,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-183::DEBUG::2012-12-09 12:10:23,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-183::DEBUG::2012-12-09 12:10:23,276::task::978::TaskManager.Task::(_decref) Task=`d4f76815-a3e9-4971-b4c9-f6a9c29dfb75`::ref 0 aborting False
>Thread-185::DEBUG::2012-12-09 12:10:23,318::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-185::DEBUG::2012-12-09 12:10:23,319::task::588::TaskManager.Task::(_updateState) Task=`f157a103-3fbc-4a91-8356-82df2fbaddc8`::moving from state init -> state preparing
>Thread-185::INFO::2012-12-09 12:10:23,319::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-185::DEBUG::2012-12-09 12:10:23,319::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-185::DEBUG::2012-12-09 12:10:23,319::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-185::INFO::2012-12-09 12:10:23,320::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-185::DEBUG::2012-12-09 12:10:23,320::task::1172::TaskManager.Task::(prepare) Task=`f157a103-3fbc-4a91-8356-82df2fbaddc8`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-185::DEBUG::2012-12-09 12:10:23,320::task::588::TaskManager.Task::(_updateState) Task=`f157a103-3fbc-4a91-8356-82df2fbaddc8`::moving from state preparing -> state finished
>Thread-185::DEBUG::2012-12-09 12:10:23,320::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-185::DEBUG::2012-12-09 12:10:23,320::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-185::DEBUG::2012-12-09 12:10:23,320::task::978::TaskManager.Task::(_decref) Task=`f157a103-3fbc-4a91-8356-82df2fbaddc8`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:23,903::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:23,904::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:10:24,020::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:24,021::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:24,021::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:24,022::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-186::DEBUG::2012-12-09 12:10:24,352::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-186::DEBUG::2012-12-09 12:10:24,353::task::588::TaskManager.Task::(_updateState) Task=`c289873e-5237-4d0d-bf85-af69585da2c6`::moving from state init -> state preparing
>Thread-186::INFO::2012-12-09 12:10:24,354::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-186::DEBUG::2012-12-09 12:10:24,354::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-186::DEBUG::2012-12-09 12:10:24,354::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-186::INFO::2012-12-09 12:10:24,354::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-186::DEBUG::2012-12-09 12:10:24,354::task::1172::TaskManager.Task::(prepare) Task=`c289873e-5237-4d0d-bf85-af69585da2c6`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-186::DEBUG::2012-12-09 12:10:24,355::task::588::TaskManager.Task::(_updateState) Task=`c289873e-5237-4d0d-bf85-af69585da2c6`::moving from state preparing -> state finished
>Thread-186::DEBUG::2012-12-09 12:10:24,355::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-186::DEBUG::2012-12-09 12:10:24,355::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-186::DEBUG::2012-12-09 12:10:24,355::task::978::TaskManager.Task::(_decref) Task=`c289873e-5237-4d0d-bf85-af69585da2c6`::ref 0 aborting False
>Thread-187::DEBUG::2012-12-09 12:10:25,380::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-187::DEBUG::2012-12-09 12:10:25,381::task::588::TaskManager.Task::(_updateState) Task=`dc7ddcc4-93bf-4d0e-b02f-78e2860ccfa6`::moving from state init -> state preparing
>Thread-187::INFO::2012-12-09 12:10:25,381::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-187::DEBUG::2012-12-09 12:10:25,381::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-187::DEBUG::2012-12-09 12:10:25,381::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-187::INFO::2012-12-09 12:10:25,382::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-187::DEBUG::2012-12-09 12:10:25,382::task::1172::TaskManager.Task::(prepare) Task=`dc7ddcc4-93bf-4d0e-b02f-78e2860ccfa6`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-187::DEBUG::2012-12-09 12:10:25,382::task::588::TaskManager.Task::(_updateState) Task=`dc7ddcc4-93bf-4d0e-b02f-78e2860ccfa6`::moving from state preparing -> state finished
>Thread-187::DEBUG::2012-12-09 12:10:25,382::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-187::DEBUG::2012-12-09 12:10:25,382::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-187::DEBUG::2012-12-09 12:10:25,382::task::978::TaskManager.Task::(_decref) Task=`dc7ddcc4-93bf-4d0e-b02f-78e2860ccfa6`::ref 0 aborting False
>Thread-189::DEBUG::2012-12-09 12:10:26,415::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-189::DEBUG::2012-12-09 12:10:26,415::task::588::TaskManager.Task::(_updateState) Task=`fc8d9c4e-23d9-47a1-93c4-1989d372d7cc`::moving from state init -> state preparing
>Thread-189::INFO::2012-12-09 12:10:26,416::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-189::DEBUG::2012-12-09 12:10:26,416::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-189::DEBUG::2012-12-09 12:10:26,416::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-189::INFO::2012-12-09 12:10:26,416::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-189::DEBUG::2012-12-09 12:10:26,416::task::1172::TaskManager.Task::(prepare) Task=`fc8d9c4e-23d9-47a1-93c4-1989d372d7cc`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-189::DEBUG::2012-12-09 12:10:26,417::task::588::TaskManager.Task::(_updateState) Task=`fc8d9c4e-23d9-47a1-93c4-1989d372d7cc`::moving from state preparing -> state finished
>Thread-189::DEBUG::2012-12-09 12:10:26,417::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-189::DEBUG::2012-12-09 12:10:26,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-189::DEBUG::2012-12-09 12:10:26,417::task::978::TaskManager.Task::(_decref) Task=`fc8d9c4e-23d9-47a1-93c4-1989d372d7cc`::ref 0 aborting False
>Thread-190::DEBUG::2012-12-09 12:10:27,442::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-190::DEBUG::2012-12-09 12:10:27,442::task::588::TaskManager.Task::(_updateState) Task=`3dffebb4-9ebe-4511-a1b7-57f5590e9a0c`::moving from state init -> state preparing
>Thread-190::INFO::2012-12-09 12:10:27,443::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-190::DEBUG::2012-12-09 12:10:27,443::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-190::DEBUG::2012-12-09 12:10:27,443::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-190::INFO::2012-12-09 12:10:27,443::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-190::DEBUG::2012-12-09 12:10:27,443::task::1172::TaskManager.Task::(prepare) Task=`3dffebb4-9ebe-4511-a1b7-57f5590e9a0c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-190::DEBUG::2012-12-09 12:10:27,444::task::588::TaskManager.Task::(_updateState) Task=`3dffebb4-9ebe-4511-a1b7-57f5590e9a0c`::moving from state preparing -> state finished
>Thread-190::DEBUG::2012-12-09 12:10:27,444::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-190::DEBUG::2012-12-09 12:10:27,444::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-190::DEBUG::2012-12-09 12:10:27,444::task::978::TaskManager.Task::(_decref) Task=`3dffebb4-9ebe-4511-a1b7-57f5590e9a0c`::ref 0 aborting False
>Thread-192::DEBUG::2012-12-09 12:10:28,475::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-192::DEBUG::2012-12-09 12:10:28,475::task::588::TaskManager.Task::(_updateState) Task=`3c7fec40-9b91-4ace-bc79-71a8cf1a66ab`::moving from state init -> state preparing
>Thread-192::INFO::2012-12-09 12:10:28,476::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-192::DEBUG::2012-12-09 12:10:28,476::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-192::DEBUG::2012-12-09 12:10:28,476::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-192::INFO::2012-12-09 12:10:28,476::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-192::DEBUG::2012-12-09 12:10:28,477::task::1172::TaskManager.Task::(prepare) Task=`3c7fec40-9b91-4ace-bc79-71a8cf1a66ab`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-192::DEBUG::2012-12-09 12:10:28,477::task::588::TaskManager.Task::(_updateState) Task=`3c7fec40-9b91-4ace-bc79-71a8cf1a66ab`::moving from state preparing -> state finished
>Thread-192::DEBUG::2012-12-09 12:10:28,477::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-192::DEBUG::2012-12-09 12:10:28,477::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-192::DEBUG::2012-12-09 12:10:28,477::task::978::TaskManager.Task::(_decref) Task=`3c7fec40-9b91-4ace-bc79-71a8cf1a66ab`::ref 0 aborting False
>Thread-194::DEBUG::2012-12-09 12:10:29,514::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-194::DEBUG::2012-12-09 12:10:29,515::task::588::TaskManager.Task::(_updateState) Task=`45f755ff-d29c-4156-909d-9239516f5452`::moving from state init -> state preparing
>Thread-194::INFO::2012-12-09 12:10:29,515::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-194::DEBUG::2012-12-09 12:10:29,515::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-194::DEBUG::2012-12-09 12:10:29,515::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-194::INFO::2012-12-09 12:10:29,516::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-194::DEBUG::2012-12-09 12:10:29,516::task::1172::TaskManager.Task::(prepare) Task=`45f755ff-d29c-4156-909d-9239516f5452`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-194::DEBUG::2012-12-09 12:10:29,516::task::588::TaskManager.Task::(_updateState) Task=`45f755ff-d29c-4156-909d-9239516f5452`::moving from state preparing -> state finished
>Thread-194::DEBUG::2012-12-09 12:10:29,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-194::DEBUG::2012-12-09 12:10:29,516::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-194::DEBUG::2012-12-09 12:10:29,517::task::978::TaskManager.Task::(_decref) Task=`45f755ff-d29c-4156-909d-9239516f5452`::ref 0 aborting False
>Thread-195::DEBUG::2012-12-09 12:10:30,546::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-195::DEBUG::2012-12-09 12:10:30,547::task::588::TaskManager.Task::(_updateState) Task=`6ea73fb2-63e3-4d3f-8f28-30c9ce7c9618`::moving from state init -> state preparing
>Thread-195::INFO::2012-12-09 12:10:30,548::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-195::DEBUG::2012-12-09 12:10:30,548::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-195::DEBUG::2012-12-09 12:10:30,549::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-195::INFO::2012-12-09 12:10:30,549::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-195::DEBUG::2012-12-09 12:10:30,549::task::1172::TaskManager.Task::(prepare) Task=`6ea73fb2-63e3-4d3f-8f28-30c9ce7c9618`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-195::DEBUG::2012-12-09 12:10:30,550::task::588::TaskManager.Task::(_updateState) Task=`6ea73fb2-63e3-4d3f-8f28-30c9ce7c9618`::moving from state preparing -> state finished
>Thread-195::DEBUG::2012-12-09 12:10:30,550::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-195::DEBUG::2012-12-09 12:10:30,551::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-195::DEBUG::2012-12-09 12:10:30,551::task::978::TaskManager.Task::(_decref) Task=`6ea73fb2-63e3-4d3f-8f28-30c9ce7c9618`::ref 0 aborting False
>Thread-197::DEBUG::2012-12-09 12:10:31,572::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-197::DEBUG::2012-12-09 12:10:31,573::task::588::TaskManager.Task::(_updateState) Task=`3e9a9a08-50ef-466d-bcb8-41a5cbfaea09`::moving from state init -> state preparing
>Thread-197::INFO::2012-12-09 12:10:31,573::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-197::DEBUG::2012-12-09 12:10:31,573::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-197::DEBUG::2012-12-09 12:10:31,574::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-197::INFO::2012-12-09 12:10:31,574::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-197::DEBUG::2012-12-09 12:10:31,574::task::1172::TaskManager.Task::(prepare) Task=`3e9a9a08-50ef-466d-bcb8-41a5cbfaea09`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-197::DEBUG::2012-12-09 12:10:31,574::task::588::TaskManager.Task::(_updateState) Task=`3e9a9a08-50ef-466d-bcb8-41a5cbfaea09`::moving from state preparing -> state finished
>Thread-197::DEBUG::2012-12-09 12:10:31,574::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-197::DEBUG::2012-12-09 12:10:31,575::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-197::DEBUG::2012-12-09 12:10:31,575::task::978::TaskManager.Task::(_decref) Task=`3e9a9a08-50ef-466d-bcb8-41a5cbfaea09`::ref 0 aborting False
>Thread-198::DEBUG::2012-12-09 12:10:32,606::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-198::DEBUG::2012-12-09 12:10:32,607::task::588::TaskManager.Task::(_updateState) Task=`ae5170c7-d5b4-4f22-8cf2-09ff1aeb76af`::moving from state init -> state preparing
>Thread-198::INFO::2012-12-09 12:10:32,607::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-198::DEBUG::2012-12-09 12:10:32,608::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-198::DEBUG::2012-12-09 12:10:32,608::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-198::INFO::2012-12-09 12:10:32,609::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-198::DEBUG::2012-12-09 12:10:32,609::task::1172::TaskManager.Task::(prepare) Task=`ae5170c7-d5b4-4f22-8cf2-09ff1aeb76af`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-198::DEBUG::2012-12-09 12:10:32,610::task::588::TaskManager.Task::(_updateState) Task=`ae5170c7-d5b4-4f22-8cf2-09ff1aeb76af`::moving from state preparing -> state finished
>Thread-198::DEBUG::2012-12-09 12:10:32,610::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-198::DEBUG::2012-12-09 12:10:32,610::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-198::DEBUG::2012-12-09 12:10:32,611::task::978::TaskManager.Task::(_decref) Task=`ae5170c7-d5b4-4f22-8cf2-09ff1aeb76af`::ref 0 aborting False
>Thread-199::DEBUG::2012-12-09 12:10:33,571::task::588::TaskManager.Task::(_updateState) Task=`9e36f465-6b8d-4021-9441-4d659d283335`::moving from state init -> state preparing
>Thread-199::INFO::2012-12-09 12:10:33,571::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-199::INFO::2012-12-09 12:10:33,572::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '90.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-199::DEBUG::2012-12-09 12:10:33,572::task::1172::TaskManager.Task::(prepare) Task=`9e36f465-6b8d-4021-9441-4d659d283335`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '90.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-199::DEBUG::2012-12-09 12:10:33,572::task::588::TaskManager.Task::(_updateState) Task=`9e36f465-6b8d-4021-9441-4d659d283335`::moving from state preparing -> state finished
>Thread-199::DEBUG::2012-12-09 12:10:33,572::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-199::DEBUG::2012-12-09 12:10:33,572::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-199::DEBUG::2012-12-09 12:10:33,572::task::978::TaskManager.Task::(_decref) Task=`9e36f465-6b8d-4021-9441-4d659d283335`::ref 0 aborting False
>Thread-201::DEBUG::2012-12-09 12:10:33,637::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-201::DEBUG::2012-12-09 12:10:33,638::task::588::TaskManager.Task::(_updateState) Task=`e0cc22be-20e6-46ba-b86c-f58ce3b54f15`::moving from state init -> state preparing
>Thread-201::INFO::2012-12-09 12:10:33,638::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-201::DEBUG::2012-12-09 12:10:33,639::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-201::DEBUG::2012-12-09 12:10:33,639::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-201::INFO::2012-12-09 12:10:33,639::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-201::DEBUG::2012-12-09 12:10:33,640::task::1172::TaskManager.Task::(prepare) Task=`e0cc22be-20e6-46ba-b86c-f58ce3b54f15`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-201::DEBUG::2012-12-09 12:10:33,640::task::588::TaskManager.Task::(_updateState) Task=`e0cc22be-20e6-46ba-b86c-f58ce3b54f15`::moving from state preparing -> state finished
>Thread-201::DEBUG::2012-12-09 12:10:33,640::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-201::DEBUG::2012-12-09 12:10:33,640::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-201::DEBUG::2012-12-09 12:10:33,640::task::978::TaskManager.Task::(_decref) Task=`e0cc22be-20e6-46ba-b86c-f58ce3b54f15`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:34,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:34,023::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:10:34,130::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:34,131::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:34,131::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:34,132::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-202::DEBUG::2012-12-09 12:10:34,675::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-202::DEBUG::2012-12-09 12:10:34,676::task::588::TaskManager.Task::(_updateState) Task=`edbef4a7-74e5-4856-9e88-ac66225abb90`::moving from state init -> state preparing
>Thread-202::INFO::2012-12-09 12:10:34,676::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-202::DEBUG::2012-12-09 12:10:34,677::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-202::DEBUG::2012-12-09 12:10:34,677::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-202::INFO::2012-12-09 12:10:34,677::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-202::DEBUG::2012-12-09 12:10:34,678::task::1172::TaskManager.Task::(prepare) Task=`edbef4a7-74e5-4856-9e88-ac66225abb90`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-202::DEBUG::2012-12-09 12:10:34,678::task::588::TaskManager.Task::(_updateState) Task=`edbef4a7-74e5-4856-9e88-ac66225abb90`::moving from state preparing -> state finished
>Thread-202::DEBUG::2012-12-09 12:10:34,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-202::DEBUG::2012-12-09 12:10:34,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-202::DEBUG::2012-12-09 12:10:34,679::task::978::TaskManager.Task::(_decref) Task=`edbef4a7-74e5-4856-9e88-ac66225abb90`::ref 0 aborting False
>Thread-203::DEBUG::2012-12-09 12:10:35,713::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-203::DEBUG::2012-12-09 12:10:35,713::task::588::TaskManager.Task::(_updateState) Task=`a9255b46-eaf5-424c-8c19-b6779f9b1132`::moving from state init -> state preparing
>Thread-203::INFO::2012-12-09 12:10:35,714::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-203::DEBUG::2012-12-09 12:10:35,714::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-203::DEBUG::2012-12-09 12:10:35,714::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-203::INFO::2012-12-09 12:10:35,714::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-203::DEBUG::2012-12-09 12:10:35,714::task::1172::TaskManager.Task::(prepare) Task=`a9255b46-eaf5-424c-8c19-b6779f9b1132`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-203::DEBUG::2012-12-09 12:10:35,715::task::588::TaskManager.Task::(_updateState) Task=`a9255b46-eaf5-424c-8c19-b6779f9b1132`::moving from state preparing -> state finished
>Thread-203::DEBUG::2012-12-09 12:10:35,715::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-203::DEBUG::2012-12-09 12:10:35,715::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-203::DEBUG::2012-12-09 12:10:35,715::task::978::TaskManager.Task::(_decref) Task=`a9255b46-eaf5-424c-8c19-b6779f9b1132`::ref 0 aborting False
>Thread-205::DEBUG::2012-12-09 12:10:36,755::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-205::DEBUG::2012-12-09 12:10:36,756::task::588::TaskManager.Task::(_updateState) Task=`c0a3147b-e52e-40b4-a86c-cedb415de9c4`::moving from state init -> state preparing
>Thread-205::INFO::2012-12-09 12:10:36,756::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-205::DEBUG::2012-12-09 12:10:36,757::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-205::DEBUG::2012-12-09 12:10:36,757::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-205::INFO::2012-12-09 12:10:36,757::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-205::DEBUG::2012-12-09 12:10:36,758::task::1172::TaskManager.Task::(prepare) Task=`c0a3147b-e52e-40b4-a86c-cedb415de9c4`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-205::DEBUG::2012-12-09 12:10:36,758::task::588::TaskManager.Task::(_updateState) Task=`c0a3147b-e52e-40b4-a86c-cedb415de9c4`::moving from state preparing -> state finished
>Thread-205::DEBUG::2012-12-09 12:10:36,758::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-205::DEBUG::2012-12-09 12:10:36,758::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-205::DEBUG::2012-12-09 12:10:36,759::task::978::TaskManager.Task::(_decref) Task=`c0a3147b-e52e-40b4-a86c-cedb415de9c4`::ref 0 aborting False
>Thread-206::DEBUG::2012-12-09 12:10:37,783::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-206::DEBUG::2012-12-09 12:10:37,784::task::588::TaskManager.Task::(_updateState) Task=`244c2b54-9d58-45d8-883f-ddc9c047d8b9`::moving from state init -> state preparing
>Thread-206::INFO::2012-12-09 12:10:37,784::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-206::DEBUG::2012-12-09 12:10:37,784::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-206::DEBUG::2012-12-09 12:10:37,785::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-206::INFO::2012-12-09 12:10:37,785::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-206::DEBUG::2012-12-09 12:10:37,785::task::1172::TaskManager.Task::(prepare) Task=`244c2b54-9d58-45d8-883f-ddc9c047d8b9`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-206::DEBUG::2012-12-09 12:10:37,785::task::588::TaskManager.Task::(_updateState) Task=`244c2b54-9d58-45d8-883f-ddc9c047d8b9`::moving from state preparing -> state finished
>Thread-206::DEBUG::2012-12-09 12:10:37,785::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-206::DEBUG::2012-12-09 12:10:37,785::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-206::DEBUG::2012-12-09 12:10:37,786::task::978::TaskManager.Task::(_decref) Task=`244c2b54-9d58-45d8-883f-ddc9c047d8b9`::ref 0 aborting False
>Thread-208::DEBUG::2012-12-09 12:10:38,821::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-208::DEBUG::2012-12-09 12:10:38,822::task::588::TaskManager.Task::(_updateState) Task=`81baab0c-66c2-438c-afb6-6467b1079a85`::moving from state init -> state preparing
>Thread-208::INFO::2012-12-09 12:10:38,823::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-208::DEBUG::2012-12-09 12:10:38,823::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-208::DEBUG::2012-12-09 12:10:38,823::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-208::INFO::2012-12-09 12:10:38,824::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-208::DEBUG::2012-12-09 12:10:38,824::task::1172::TaskManager.Task::(prepare) Task=`81baab0c-66c2-438c-afb6-6467b1079a85`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-208::DEBUG::2012-12-09 12:10:38,825::task::588::TaskManager.Task::(_updateState) Task=`81baab0c-66c2-438c-afb6-6467b1079a85`::moving from state preparing -> state finished
>Thread-208::DEBUG::2012-12-09 12:10:38,825::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-208::DEBUG::2012-12-09 12:10:38,826::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-208::DEBUG::2012-12-09 12:10:38,826::task::978::TaskManager.Task::(_decref) Task=`81baab0c-66c2-438c-afb6-6467b1079a85`::ref 0 aborting False
>Thread-210::DEBUG::2012-12-09 12:10:39,857::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-210::DEBUG::2012-12-09 12:10:39,858::task::588::TaskManager.Task::(_updateState) Task=`234cffc5-079e-4081-8fcc-37cc261a2897`::moving from state init -> state preparing
>Thread-210::INFO::2012-12-09 12:10:39,859::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-210::DEBUG::2012-12-09 12:10:39,859::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-210::DEBUG::2012-12-09 12:10:39,859::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-210::INFO::2012-12-09 12:10:39,860::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-210::DEBUG::2012-12-09 12:10:39,860::task::1172::TaskManager.Task::(prepare) Task=`234cffc5-079e-4081-8fcc-37cc261a2897`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-210::DEBUG::2012-12-09 12:10:39,861::task::588::TaskManager.Task::(_updateState) Task=`234cffc5-079e-4081-8fcc-37cc261a2897`::moving from state preparing -> state finished
>Thread-210::DEBUG::2012-12-09 12:10:39,861::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-210::DEBUG::2012-12-09 12:10:39,862::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-210::DEBUG::2012-12-09 12:10:39,862::task::978::TaskManager.Task::(_decref) Task=`234cffc5-079e-4081-8fcc-37cc261a2897`::ref 0 aborting False
>Thread-211::DEBUG::2012-12-09 12:10:40,893::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-211::DEBUG::2012-12-09 12:10:40,894::task::588::TaskManager.Task::(_updateState) Task=`9edf78f1-0e4f-4ba4-9cd0-dc8cecba2605`::moving from state init -> state preparing
>Thread-211::INFO::2012-12-09 12:10:40,895::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-211::DEBUG::2012-12-09 12:10:40,895::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-211::DEBUG::2012-12-09 12:10:40,895::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-211::INFO::2012-12-09 12:10:40,896::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-211::DEBUG::2012-12-09 12:10:40,896::task::1172::TaskManager.Task::(prepare) Task=`9edf78f1-0e4f-4ba4-9cd0-dc8cecba2605`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-211::DEBUG::2012-12-09 12:10:40,897::task::588::TaskManager.Task::(_updateState) Task=`9edf78f1-0e4f-4ba4-9cd0-dc8cecba2605`::moving from state preparing -> state finished
>Thread-211::DEBUG::2012-12-09 12:10:40,897::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-211::DEBUG::2012-12-09 12:10:40,897::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-211::DEBUG::2012-12-09 12:10:40,898::task::978::TaskManager.Task::(_decref) Task=`9edf78f1-0e4f-4ba4-9cd0-dc8cecba2605`::ref 0 aborting False
>Thread-213::DEBUG::2012-12-09 12:10:41,920::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-213::DEBUG::2012-12-09 12:10:41,920::task::588::TaskManager.Task::(_updateState) Task=`f7445c38-01c2-4e6e-8526-08396865ceff`::moving from state init -> state preparing
>Thread-213::INFO::2012-12-09 12:10:41,921::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-213::DEBUG::2012-12-09 12:10:41,921::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-213::DEBUG::2012-12-09 12:10:41,921::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-213::INFO::2012-12-09 12:10:41,921::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-213::DEBUG::2012-12-09 12:10:41,921::task::1172::TaskManager.Task::(prepare) Task=`f7445c38-01c2-4e6e-8526-08396865ceff`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-213::DEBUG::2012-12-09 12:10:41,922::task::588::TaskManager.Task::(_updateState) Task=`f7445c38-01c2-4e6e-8526-08396865ceff`::moving from state preparing -> state finished
>Thread-213::DEBUG::2012-12-09 12:10:41,922::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-213::DEBUG::2012-12-09 12:10:41,922::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-213::DEBUG::2012-12-09 12:10:41,922::task::978::TaskManager.Task::(_decref) Task=`f7445c38-01c2-4e6e-8526-08396865ceff`::ref 0 aborting False
>Thread-214::DEBUG::2012-12-09 12:10:42,944::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-214::DEBUG::2012-12-09 12:10:42,945::task::588::TaskManager.Task::(_updateState) Task=`c14678b0-639e-449a-9604-79c3763dd5d2`::moving from state init -> state preparing
>Thread-214::INFO::2012-12-09 12:10:42,945::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-214::DEBUG::2012-12-09 12:10:42,945::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-214::DEBUG::2012-12-09 12:10:42,946::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-214::INFO::2012-12-09 12:10:42,946::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-214::DEBUG::2012-12-09 12:10:42,946::task::1172::TaskManager.Task::(prepare) Task=`c14678b0-639e-449a-9604-79c3763dd5d2`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-214::DEBUG::2012-12-09 12:10:42,946::task::588::TaskManager.Task::(_updateState) Task=`c14678b0-639e-449a-9604-79c3763dd5d2`::moving from state preparing -> state finished
>Thread-214::DEBUG::2012-12-09 12:10:42,946::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-214::DEBUG::2012-12-09 12:10:42,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-214::DEBUG::2012-12-09 12:10:42,947::task::978::TaskManager.Task::(_decref) Task=`c14678b0-639e-449a-9604-79c3763dd5d2`::ref 0 aborting False
>Thread-215::DEBUG::2012-12-09 12:10:43,917::task::588::TaskManager.Task::(_updateState) Task=`bcb19094-ff3a-40f9-8f5c-a17d57eababa`::moving from state init -> state preparing
>Thread-215::INFO::2012-12-09 12:10:43,918::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-215::INFO::2012-12-09 12:10:43,918::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '100.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-215::DEBUG::2012-12-09 12:10:43,919::task::1172::TaskManager.Task::(prepare) Task=`bcb19094-ff3a-40f9-8f5c-a17d57eababa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '100.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-215::DEBUG::2012-12-09 12:10:43,919::task::588::TaskManager.Task::(_updateState) Task=`bcb19094-ff3a-40f9-8f5c-a17d57eababa`::moving from state preparing -> state finished
>Thread-215::DEBUG::2012-12-09 12:10:43,919::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-215::DEBUG::2012-12-09 12:10:43,920::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-215::DEBUG::2012-12-09 12:10:43,920::task::978::TaskManager.Task::(_decref) Task=`bcb19094-ff3a-40f9-8f5c-a17d57eababa`::ref 0 aborting False
>Thread-217::DEBUG::2012-12-09 12:10:43,972::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-217::DEBUG::2012-12-09 12:10:43,973::task::588::TaskManager.Task::(_updateState) Task=`b1d7fc56-665d-497e-a87c-07dfbdd6f3ec`::moving from state init -> state preparing
>Thread-217::INFO::2012-12-09 12:10:43,973::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-217::DEBUG::2012-12-09 12:10:43,973::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-217::DEBUG::2012-12-09 12:10:43,973::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-217::INFO::2012-12-09 12:10:43,973::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-217::DEBUG::2012-12-09 12:10:43,974::task::1172::TaskManager.Task::(prepare) Task=`b1d7fc56-665d-497e-a87c-07dfbdd6f3ec`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-217::DEBUG::2012-12-09 12:10:43,974::task::588::TaskManager.Task::(_updateState) Task=`b1d7fc56-665d-497e-a87c-07dfbdd6f3ec`::moving from state preparing -> state finished
>Thread-217::DEBUG::2012-12-09 12:10:43,974::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-217::DEBUG::2012-12-09 12:10:43,974::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-217::DEBUG::2012-12-09 12:10:43,974::task::978::TaskManager.Task::(_decref) Task=`b1d7fc56-665d-497e-a87c-07dfbdd6f3ec`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:44,133::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:44,134::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:10:44,240::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:44,241::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:44,241::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:44,242::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-218::DEBUG::2012-12-09 12:10:45,007::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-218::DEBUG::2012-12-09 12:10:45,008::task::588::TaskManager.Task::(_updateState) Task=`487c2171-d1e8-4421-bb88-bc6557e230c8`::moving from state init -> state preparing
>Thread-218::INFO::2012-12-09 12:10:45,008::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-218::DEBUG::2012-12-09 12:10:45,008::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-218::DEBUG::2012-12-09 12:10:45,009::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-218::INFO::2012-12-09 12:10:45,009::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-218::DEBUG::2012-12-09 12:10:45,009::task::1172::TaskManager.Task::(prepare) Task=`487c2171-d1e8-4421-bb88-bc6557e230c8`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-218::DEBUG::2012-12-09 12:10:45,009::task::588::TaskManager.Task::(_updateState) Task=`487c2171-d1e8-4421-bb88-bc6557e230c8`::moving from state preparing -> state finished
>Thread-218::DEBUG::2012-12-09 12:10:45,009::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-218::DEBUG::2012-12-09 12:10:45,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-218::DEBUG::2012-12-09 12:10:45,010::task::978::TaskManager.Task::(_decref) Task=`487c2171-d1e8-4421-bb88-bc6557e230c8`::ref 0 aborting False
>Thread-219::DEBUG::2012-12-09 12:10:46,041::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-219::DEBUG::2012-12-09 12:10:46,041::task::588::TaskManager.Task::(_updateState) Task=`74988328-10d3-412b-8941-78ad643657bb`::moving from state init -> state preparing
>Thread-219::INFO::2012-12-09 12:10:46,042::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-219::DEBUG::2012-12-09 12:10:46,042::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-219::DEBUG::2012-12-09 12:10:46,042::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-219::INFO::2012-12-09 12:10:46,042::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-219::DEBUG::2012-12-09 12:10:46,042::task::1172::TaskManager.Task::(prepare) Task=`74988328-10d3-412b-8941-78ad643657bb`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-219::DEBUG::2012-12-09 12:10:46,043::task::588::TaskManager.Task::(_updateState) Task=`74988328-10d3-412b-8941-78ad643657bb`::moving from state preparing -> state finished
>Thread-219::DEBUG::2012-12-09 12:10:46,043::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-219::DEBUG::2012-12-09 12:10:46,043::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-219::DEBUG::2012-12-09 12:10:46,043::task::978::TaskManager.Task::(_decref) Task=`74988328-10d3-412b-8941-78ad643657bb`::ref 0 aborting False
>Thread-221::DEBUG::2012-12-09 12:10:47,074::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-221::DEBUG::2012-12-09 12:10:47,075::task::588::TaskManager.Task::(_updateState) Task=`e7b8010e-a19f-48fd-baeb-069c073c7ede`::moving from state init -> state preparing
>Thread-221::INFO::2012-12-09 12:10:47,075::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-221::DEBUG::2012-12-09 12:10:47,075::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-221::DEBUG::2012-12-09 12:10:47,076::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-221::INFO::2012-12-09 12:10:47,076::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-221::DEBUG::2012-12-09 12:10:47,077::task::1172::TaskManager.Task::(prepare) Task=`e7b8010e-a19f-48fd-baeb-069c073c7ede`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-221::DEBUG::2012-12-09 12:10:47,077::task::588::TaskManager.Task::(_updateState) Task=`e7b8010e-a19f-48fd-baeb-069c073c7ede`::moving from state preparing -> state finished
>Thread-221::DEBUG::2012-12-09 12:10:47,078::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-221::DEBUG::2012-12-09 12:10:47,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-221::DEBUG::2012-12-09 12:10:47,078::task::978::TaskManager.Task::(_decref) Task=`e7b8010e-a19f-48fd-baeb-069c073c7ede`::ref 0 aborting False
>Thread-223::DEBUG::2012-12-09 12:10:48,105::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-223::DEBUG::2012-12-09 12:10:48,106::task::588::TaskManager.Task::(_updateState) Task=`f986e528-b3df-4fc7-bf04-cfc3f95f18a1`::moving from state init -> state preparing
>Thread-223::INFO::2012-12-09 12:10:48,106::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-223::DEBUG::2012-12-09 12:10:48,106::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-223::DEBUG::2012-12-09 12:10:48,106::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-223::INFO::2012-12-09 12:10:48,107::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-223::DEBUG::2012-12-09 12:10:48,107::task::1172::TaskManager.Task::(prepare) Task=`f986e528-b3df-4fc7-bf04-cfc3f95f18a1`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-223::DEBUG::2012-12-09 12:10:48,107::task::588::TaskManager.Task::(_updateState) Task=`f986e528-b3df-4fc7-bf04-cfc3f95f18a1`::moving from state preparing -> state finished
>Thread-223::DEBUG::2012-12-09 12:10:48,107::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-223::DEBUG::2012-12-09 12:10:48,107::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-223::DEBUG::2012-12-09 12:10:48,107::task::978::TaskManager.Task::(_decref) Task=`f986e528-b3df-4fc7-bf04-cfc3f95f18a1`::ref 0 aborting False
>Thread-224::DEBUG::2012-12-09 12:10:49,136::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-224::DEBUG::2012-12-09 12:10:49,137::task::588::TaskManager.Task::(_updateState) Task=`1cca7e2f-0582-4693-90af-6fe7b357015e`::moving from state init -> state preparing
>Thread-224::INFO::2012-12-09 12:10:49,138::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-224::DEBUG::2012-12-09 12:10:49,138::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-224::DEBUG::2012-12-09 12:10:49,139::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-224::INFO::2012-12-09 12:10:49,139::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-224::DEBUG::2012-12-09 12:10:49,139::task::1172::TaskManager.Task::(prepare) Task=`1cca7e2f-0582-4693-90af-6fe7b357015e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-224::DEBUG::2012-12-09 12:10:49,140::task::588::TaskManager.Task::(_updateState) Task=`1cca7e2f-0582-4693-90af-6fe7b357015e`::moving from state preparing -> state finished
>Thread-224::DEBUG::2012-12-09 12:10:49,140::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-224::DEBUG::2012-12-09 12:10:49,140::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-224::DEBUG::2012-12-09 12:10:49,140::task::978::TaskManager.Task::(_decref) Task=`1cca7e2f-0582-4693-90af-6fe7b357015e`::ref 0 aborting False
>Thread-226::DEBUG::2012-12-09 12:10:50,167::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-226::DEBUG::2012-12-09 12:10:50,167::task::588::TaskManager.Task::(_updateState) Task=`4e57bf8a-10f9-4ebd-891d-2d695122d079`::moving from state init -> state preparing
>Thread-226::INFO::2012-12-09 12:10:50,168::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-226::DEBUG::2012-12-09 12:10:50,168::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-226::DEBUG::2012-12-09 12:10:50,169::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-226::INFO::2012-12-09 12:10:50,169::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-226::DEBUG::2012-12-09 12:10:50,170::task::1172::TaskManager.Task::(prepare) Task=`4e57bf8a-10f9-4ebd-891d-2d695122d079`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-226::DEBUG::2012-12-09 12:10:50,170::task::588::TaskManager.Task::(_updateState) Task=`4e57bf8a-10f9-4ebd-891d-2d695122d079`::moving from state preparing -> state finished
>Thread-226::DEBUG::2012-12-09 12:10:50,170::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-226::DEBUG::2012-12-09 12:10:50,171::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-226::DEBUG::2012-12-09 12:10:50,171::task::978::TaskManager.Task::(_decref) Task=`4e57bf8a-10f9-4ebd-891d-2d695122d079`::ref 0 aborting False
>Thread-227::DEBUG::2012-12-09 12:10:51,193::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-227::DEBUG::2012-12-09 12:10:51,194::task::588::TaskManager.Task::(_updateState) Task=`8dea0237-f66e-4d98-b12c-938aa4a6ba47`::moving from state init -> state preparing
>Thread-227::INFO::2012-12-09 12:10:51,194::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-227::DEBUG::2012-12-09 12:10:51,194::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-227::DEBUG::2012-12-09 12:10:51,194::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-227::INFO::2012-12-09 12:10:51,194::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-227::DEBUG::2012-12-09 12:10:51,195::task::1172::TaskManager.Task::(prepare) Task=`8dea0237-f66e-4d98-b12c-938aa4a6ba47`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-227::DEBUG::2012-12-09 12:10:51,195::task::588::TaskManager.Task::(_updateState) Task=`8dea0237-f66e-4d98-b12c-938aa4a6ba47`::moving from state preparing -> state finished
>Thread-227::DEBUG::2012-12-09 12:10:51,195::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-227::DEBUG::2012-12-09 12:10:51,195::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-227::DEBUG::2012-12-09 12:10:51,195::task::978::TaskManager.Task::(_decref) Task=`8dea0237-f66e-4d98-b12c-938aa4a6ba47`::ref 0 aborting False
>Thread-229::DEBUG::2012-12-09 12:10:52,234::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-229::DEBUG::2012-12-09 12:10:52,235::task::588::TaskManager.Task::(_updateState) Task=`a4646daf-a0e5-4529-b70f-0304298ca90b`::moving from state init -> state preparing
>Thread-229::INFO::2012-12-09 12:10:52,235::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-229::DEBUG::2012-12-09 12:10:52,235::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-229::DEBUG::2012-12-09 12:10:52,235::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-229::INFO::2012-12-09 12:10:52,235::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-229::DEBUG::2012-12-09 12:10:52,236::task::1172::TaskManager.Task::(prepare) Task=`a4646daf-a0e5-4529-b70f-0304298ca90b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-229::DEBUG::2012-12-09 12:10:52,236::task::588::TaskManager.Task::(_updateState) Task=`a4646daf-a0e5-4529-b70f-0304298ca90b`::moving from state preparing -> state finished
>Thread-229::DEBUG::2012-12-09 12:10:52,236::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-229::DEBUG::2012-12-09 12:10:52,236::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-229::DEBUG::2012-12-09 12:10:52,236::task::978::TaskManager.Task::(_decref) Task=`a4646daf-a0e5-4529-b70f-0304298ca90b`::ref 0 aborting False
>Thread-230::DEBUG::2012-12-09 12:10:53,274::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-230::DEBUG::2012-12-09 12:10:53,275::task::588::TaskManager.Task::(_updateState) Task=`92b1643a-1c35-43a4-abba-bd696146cb73`::moving from state init -> state preparing
>Thread-230::INFO::2012-12-09 12:10:53,275::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-230::DEBUG::2012-12-09 12:10:53,276::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-230::DEBUG::2012-12-09 12:10:53,276::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-230::INFO::2012-12-09 12:10:53,277::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-230::DEBUG::2012-12-09 12:10:53,277::task::1172::TaskManager.Task::(prepare) Task=`92b1643a-1c35-43a4-abba-bd696146cb73`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-230::DEBUG::2012-12-09 12:10:53,278::task::588::TaskManager.Task::(_updateState) Task=`92b1643a-1c35-43a4-abba-bd696146cb73`::moving from state preparing -> state finished
>Thread-230::DEBUG::2012-12-09 12:10:53,278::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-230::DEBUG::2012-12-09 12:10:53,278::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-230::DEBUG::2012-12-09 12:10:53,279::task::978::TaskManager.Task::(_decref) Task=`92b1643a-1c35-43a4-abba-bd696146cb73`::ref 0 aborting False
>Thread-231::DEBUG::2012-12-09 12:10:54,180::task::588::TaskManager.Task::(_updateState) Task=`b5f59d85-c119-409e-baf8-402e6348693f`::moving from state init -> state preparing
>Thread-231::INFO::2012-12-09 12:10:54,180::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-231::INFO::2012-12-09 12:10:54,181::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '111.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-231::DEBUG::2012-12-09 12:10:54,181::task::1172::TaskManager.Task::(prepare) Task=`b5f59d85-c119-409e-baf8-402e6348693f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '111.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-231::DEBUG::2012-12-09 12:10:54,181::task::588::TaskManager.Task::(_updateState) Task=`b5f59d85-c119-409e-baf8-402e6348693f`::moving from state preparing -> state finished
>Thread-231::DEBUG::2012-12-09 12:10:54,181::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-231::DEBUG::2012-12-09 12:10:54,181::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-231::DEBUG::2012-12-09 12:10:54,182::task::978::TaskManager.Task::(_decref) Task=`b5f59d85-c119-409e-baf8-402e6348693f`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:54,242::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:10:54,243::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-233::DEBUG::2012-12-09 12:10:54,301::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-233::DEBUG::2012-12-09 12:10:54,302::task::588::TaskManager.Task::(_updateState) Task=`cfcc3f40-b3a0-4d51-b4d9-f490faab2bac`::moving from state init -> state preparing
>Thread-233::INFO::2012-12-09 12:10:54,303::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-233::DEBUG::2012-12-09 12:10:54,303::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-233::DEBUG::2012-12-09 12:10:54,303::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-233::INFO::2012-12-09 12:10:54,303::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-233::DEBUG::2012-12-09 12:10:54,304::task::1172::TaskManager.Task::(prepare) Task=`cfcc3f40-b3a0-4d51-b4d9-f490faab2bac`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-233::DEBUG::2012-12-09 12:10:54,304::task::588::TaskManager.Task::(_updateState) Task=`cfcc3f40-b3a0-4d51-b4d9-f490faab2bac`::moving from state preparing -> state finished
>Thread-233::DEBUG::2012-12-09 12:10:54,304::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-233::DEBUG::2012-12-09 12:10:54,305::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-233::DEBUG::2012-12-09 12:10:54,305::task::978::TaskManager.Task::(_decref) Task=`cfcc3f40-b3a0-4d51-b4d9-f490faab2bac`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:10:54,353::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:10:54,355::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:10:54,356::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:10:54,357::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-234::DEBUG::2012-12-09 12:10:55,329::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-234::DEBUG::2012-12-09 12:10:55,330::task::588::TaskManager.Task::(_updateState) Task=`6a4ed17b-0aa8-45f7-b371-1d3141054bf4`::moving from state init -> state preparing
>Thread-234::INFO::2012-12-09 12:10:55,330::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-234::DEBUG::2012-12-09 12:10:55,330::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-234::DEBUG::2012-12-09 12:10:55,331::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-234::INFO::2012-12-09 12:10:55,331::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-234::DEBUG::2012-12-09 12:10:55,331::task::1172::TaskManager.Task::(prepare) Task=`6a4ed17b-0aa8-45f7-b371-1d3141054bf4`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-234::DEBUG::2012-12-09 12:10:55,331::task::588::TaskManager.Task::(_updateState) Task=`6a4ed17b-0aa8-45f7-b371-1d3141054bf4`::moving from state preparing -> state finished
>Thread-234::DEBUG::2012-12-09 12:10:55,331::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-234::DEBUG::2012-12-09 12:10:55,332::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-234::DEBUG::2012-12-09 12:10:55,332::task::978::TaskManager.Task::(_decref) Task=`6a4ed17b-0aa8-45f7-b371-1d3141054bf4`::ref 0 aborting False
>Thread-236::DEBUG::2012-12-09 12:10:56,362::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-236::DEBUG::2012-12-09 12:10:56,363::task::588::TaskManager.Task::(_updateState) Task=`6028c0e4-e606-4671-bff1-407dfde9f808`::moving from state init -> state preparing
>Thread-236::INFO::2012-12-09 12:10:56,364::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-236::DEBUG::2012-12-09 12:10:56,364::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-236::DEBUG::2012-12-09 12:10:56,364::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-236::INFO::2012-12-09 12:10:56,365::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-236::DEBUG::2012-12-09 12:10:56,365::task::1172::TaskManager.Task::(prepare) Task=`6028c0e4-e606-4671-bff1-407dfde9f808`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-236::DEBUG::2012-12-09 12:10:56,366::task::588::TaskManager.Task::(_updateState) Task=`6028c0e4-e606-4671-bff1-407dfde9f808`::moving from state preparing -> state finished
>Thread-236::DEBUG::2012-12-09 12:10:56,366::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-236::DEBUG::2012-12-09 12:10:56,366::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-236::DEBUG::2012-12-09 12:10:56,367::task::978::TaskManager.Task::(_decref) Task=`6028c0e4-e606-4671-bff1-407dfde9f808`::ref 0 aborting False
>Thread-237::DEBUG::2012-12-09 12:10:57,402::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-237::DEBUG::2012-12-09 12:10:57,403::task::588::TaskManager.Task::(_updateState) Task=`8ccfe3c5-587e-48fd-8d3a-e2cdc305699b`::moving from state init -> state preparing
>Thread-237::INFO::2012-12-09 12:10:57,404::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-237::DEBUG::2012-12-09 12:10:57,404::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-237::DEBUG::2012-12-09 12:10:57,405::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-237::INFO::2012-12-09 12:10:57,405::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-237::DEBUG::2012-12-09 12:10:57,405::task::1172::TaskManager.Task::(prepare) Task=`8ccfe3c5-587e-48fd-8d3a-e2cdc305699b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-237::DEBUG::2012-12-09 12:10:57,406::task::588::TaskManager.Task::(_updateState) Task=`8ccfe3c5-587e-48fd-8d3a-e2cdc305699b`::moving from state preparing -> state finished
>Thread-237::DEBUG::2012-12-09 12:10:57,406::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-237::DEBUG::2012-12-09 12:10:57,407::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-237::DEBUG::2012-12-09 12:10:57,407::task::978::TaskManager.Task::(_decref) Task=`8ccfe3c5-587e-48fd-8d3a-e2cdc305699b`::ref 0 aborting False
>Thread-239::DEBUG::2012-12-09 12:10:58,442::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-239::DEBUG::2012-12-09 12:10:58,443::task::588::TaskManager.Task::(_updateState) Task=`567f14af-15e9-4787-b942-eea18e10792d`::moving from state init -> state preparing
>Thread-239::INFO::2012-12-09 12:10:58,444::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-239::DEBUG::2012-12-09 12:10:58,444::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-239::DEBUG::2012-12-09 12:10:58,445::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-239::INFO::2012-12-09 12:10:58,445::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-239::DEBUG::2012-12-09 12:10:58,445::task::1172::TaskManager.Task::(prepare) Task=`567f14af-15e9-4787-b942-eea18e10792d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-239::DEBUG::2012-12-09 12:10:58,446::task::588::TaskManager.Task::(_updateState) Task=`567f14af-15e9-4787-b942-eea18e10792d`::moving from state preparing -> state finished
>Thread-239::DEBUG::2012-12-09 12:10:58,446::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-239::DEBUG::2012-12-09 12:10:58,447::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-239::DEBUG::2012-12-09 12:10:58,447::task::978::TaskManager.Task::(_decref) Task=`567f14af-15e9-4787-b942-eea18e10792d`::ref 0 aborting False
>Thread-240::DEBUG::2012-12-09 12:10:59,481::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-240::DEBUG::2012-12-09 12:10:59,482::task::588::TaskManager.Task::(_updateState) Task=`ba47586d-2b81-493b-a214-00867bf26e56`::moving from state init -> state preparing
>Thread-240::INFO::2012-12-09 12:10:59,482::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-240::DEBUG::2012-12-09 12:10:59,482::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-240::DEBUG::2012-12-09 12:10:59,483::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-240::INFO::2012-12-09 12:10:59,483::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-240::DEBUG::2012-12-09 12:10:59,483::task::1172::TaskManager.Task::(prepare) Task=`ba47586d-2b81-493b-a214-00867bf26e56`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-240::DEBUG::2012-12-09 12:10:59,483::task::588::TaskManager.Task::(_updateState) Task=`ba47586d-2b81-493b-a214-00867bf26e56`::moving from state preparing -> state finished
>Thread-240::DEBUG::2012-12-09 12:10:59,483::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-240::DEBUG::2012-12-09 12:10:59,483::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-240::DEBUG::2012-12-09 12:10:59,484::task::978::TaskManager.Task::(_decref) Task=`ba47586d-2b81-493b-a214-00867bf26e56`::ref 0 aborting False
>Thread-242::DEBUG::2012-12-09 12:11:00,513::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-242::DEBUG::2012-12-09 12:11:00,513::task::588::TaskManager.Task::(_updateState) Task=`679df7cf-e85d-4fb2-ae35-d3f980d19351`::moving from state init -> state preparing
>Thread-242::INFO::2012-12-09 12:11:00,514::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-242::DEBUG::2012-12-09 12:11:00,514::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-242::DEBUG::2012-12-09 12:11:00,514::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-242::INFO::2012-12-09 12:11:00,514::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-242::DEBUG::2012-12-09 12:11:00,514::task::1172::TaskManager.Task::(prepare) Task=`679df7cf-e85d-4fb2-ae35-d3f980d19351`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-242::DEBUG::2012-12-09 12:11:00,515::task::588::TaskManager.Task::(_updateState) Task=`679df7cf-e85d-4fb2-ae35-d3f980d19351`::moving from state preparing -> state finished
>Thread-242::DEBUG::2012-12-09 12:11:00,515::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-242::DEBUG::2012-12-09 12:11:00,515::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-242::DEBUG::2012-12-09 12:11:00,515::task::978::TaskManager.Task::(_decref) Task=`679df7cf-e85d-4fb2-ae35-d3f980d19351`::ref 0 aborting False
>Thread-243::DEBUG::2012-12-09 12:11:01,546::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-243::DEBUG::2012-12-09 12:11:01,546::task::588::TaskManager.Task::(_updateState) Task=`ebcc9e0d-7131-4ae8-9b31-622c338cd814`::moving from state init -> state preparing
>Thread-243::INFO::2012-12-09 12:11:01,546::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-243::DEBUG::2012-12-09 12:11:01,547::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-243::DEBUG::2012-12-09 12:11:01,547::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-243::INFO::2012-12-09 12:11:01,547::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-243::DEBUG::2012-12-09 12:11:01,547::task::1172::TaskManager.Task::(prepare) Task=`ebcc9e0d-7131-4ae8-9b31-622c338cd814`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-243::DEBUG::2012-12-09 12:11:01,547::task::588::TaskManager.Task::(_updateState) Task=`ebcc9e0d-7131-4ae8-9b31-622c338cd814`::moving from state preparing -> state finished
>Thread-243::DEBUG::2012-12-09 12:11:01,547::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-243::DEBUG::2012-12-09 12:11:01,548::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-243::DEBUG::2012-12-09 12:11:01,548::task::978::TaskManager.Task::(_decref) Task=`ebcc9e0d-7131-4ae8-9b31-622c338cd814`::ref 0 aborting False
>Thread-245::DEBUG::2012-12-09 12:11:02,569::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-245::DEBUG::2012-12-09 12:11:02,570::task::588::TaskManager.Task::(_updateState) Task=`884838e8-0eb2-4b7b-a93f-8e71723051ef`::moving from state init -> state preparing
>Thread-245::INFO::2012-12-09 12:11:02,570::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-245::DEBUG::2012-12-09 12:11:02,570::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-245::DEBUG::2012-12-09 12:11:02,571::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-245::INFO::2012-12-09 12:11:02,571::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-245::DEBUG::2012-12-09 12:11:02,571::task::1172::TaskManager.Task::(prepare) Task=`884838e8-0eb2-4b7b-a93f-8e71723051ef`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-245::DEBUG::2012-12-09 12:11:02,571::task::588::TaskManager.Task::(_updateState) Task=`884838e8-0eb2-4b7b-a93f-8e71723051ef`::moving from state preparing -> state finished
>Thread-245::DEBUG::2012-12-09 12:11:02,571::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-245::DEBUG::2012-12-09 12:11:02,572::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-245::DEBUG::2012-12-09 12:11:02,572::task::978::TaskManager.Task::(_decref) Task=`884838e8-0eb2-4b7b-a93f-8e71723051ef`::ref 0 aborting False
>Thread-246::DEBUG::2012-12-09 12:11:03,597::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-246::DEBUG::2012-12-09 12:11:03,598::task::588::TaskManager.Task::(_updateState) Task=`07631548-ea82-4dc2-a30f-100bfd92a5cf`::moving from state init -> state preparing
>Thread-246::INFO::2012-12-09 12:11:03,599::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-246::DEBUG::2012-12-09 12:11:03,599::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-246::DEBUG::2012-12-09 12:11:03,599::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-246::INFO::2012-12-09 12:11:03,600::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-246::DEBUG::2012-12-09 12:11:03,600::task::1172::TaskManager.Task::(prepare) Task=`07631548-ea82-4dc2-a30f-100bfd92a5cf`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-246::DEBUG::2012-12-09 12:11:03,600::task::588::TaskManager.Task::(_updateState) Task=`07631548-ea82-4dc2-a30f-100bfd92a5cf`::moving from state preparing -> state finished
>Thread-246::DEBUG::2012-12-09 12:11:03,600::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-246::DEBUG::2012-12-09 12:11:03,600::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-246::DEBUG::2012-12-09 12:11:03,600::task::978::TaskManager.Task::(_decref) Task=`07631548-ea82-4dc2-a30f-100bfd92a5cf`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:04,358::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:04,358::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-247::DEBUG::2012-12-09 12:11:04,429::task::588::TaskManager.Task::(_updateState) Task=`41b7a051-a34d-44c5-8367-71d87a6df773`::moving from state init -> state preparing
>Thread-247::INFO::2012-12-09 12:11:04,429::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-247::INFO::2012-12-09 12:11:04,430::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '121.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-247::DEBUG::2012-12-09 12:11:04,430::task::1172::TaskManager.Task::(prepare) Task=`41b7a051-a34d-44c5-8367-71d87a6df773`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '121.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-247::DEBUG::2012-12-09 12:11:04,430::task::588::TaskManager.Task::(_updateState) Task=`41b7a051-a34d-44c5-8367-71d87a6df773`::moving from state preparing -> state finished
>Thread-247::DEBUG::2012-12-09 12:11:04,430::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-247::DEBUG::2012-12-09 12:11:04,430::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-247::DEBUG::2012-12-09 12:11:04,431::task::978::TaskManager.Task::(_decref) Task=`41b7a051-a34d-44c5-8367-71d87a6df773`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:04,459::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:04,460::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:04,460::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:04,461::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-249::DEBUG::2012-12-09 12:11:04,622::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-249::DEBUG::2012-12-09 12:11:04,623::task::588::TaskManager.Task::(_updateState) Task=`77888e1d-d69b-464d-84a4-f1850d3a63d6`::moving from state init -> state preparing
>Thread-249::INFO::2012-12-09 12:11:04,623::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-249::DEBUG::2012-12-09 12:11:04,623::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-249::DEBUG::2012-12-09 12:11:04,623::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-249::INFO::2012-12-09 12:11:04,624::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-249::DEBUG::2012-12-09 12:11:04,624::task::1172::TaskManager.Task::(prepare) Task=`77888e1d-d69b-464d-84a4-f1850d3a63d6`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-249::DEBUG::2012-12-09 12:11:04,624::task::588::TaskManager.Task::(_updateState) Task=`77888e1d-d69b-464d-84a4-f1850d3a63d6`::moving from state preparing -> state finished
>Thread-249::DEBUG::2012-12-09 12:11:04,624::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-249::DEBUG::2012-12-09 12:11:04,624::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-249::DEBUG::2012-12-09 12:11:04,625::task::978::TaskManager.Task::(_decref) Task=`77888e1d-d69b-464d-84a4-f1850d3a63d6`::ref 0 aborting False
>Thread-250::DEBUG::2012-12-09 12:11:05,655::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-250::DEBUG::2012-12-09 12:11:05,656::task::588::TaskManager.Task::(_updateState) Task=`c3f1029a-61db-4711-868b-fa2c8046ff9d`::moving from state init -> state preparing
>Thread-250::INFO::2012-12-09 12:11:05,657::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-250::DEBUG::2012-12-09 12:11:05,657::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-250::DEBUG::2012-12-09 12:11:05,657::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-250::INFO::2012-12-09 12:11:05,658::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-250::DEBUG::2012-12-09 12:11:05,658::task::1172::TaskManager.Task::(prepare) Task=`c3f1029a-61db-4711-868b-fa2c8046ff9d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-250::DEBUG::2012-12-09 12:11:05,659::task::588::TaskManager.Task::(_updateState) Task=`c3f1029a-61db-4711-868b-fa2c8046ff9d`::moving from state preparing -> state finished
>Thread-250::DEBUG::2012-12-09 12:11:05,659::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-250::DEBUG::2012-12-09 12:11:05,659::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-250::DEBUG::2012-12-09 12:11:05,660::task::978::TaskManager.Task::(_decref) Task=`c3f1029a-61db-4711-868b-fa2c8046ff9d`::ref 0 aborting False
>Thread-252::DEBUG::2012-12-09 12:11:06,681::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-252::DEBUG::2012-12-09 12:11:06,682::task::588::TaskManager.Task::(_updateState) Task=`e0d2b09b-ff0f-4c33-914e-7691c27fd2ac`::moving from state init -> state preparing
>Thread-252::INFO::2012-12-09 12:11:06,682::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-252::DEBUG::2012-12-09 12:11:06,682::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-252::DEBUG::2012-12-09 12:11:06,682::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-252::INFO::2012-12-09 12:11:06,683::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-252::DEBUG::2012-12-09 12:11:06,683::task::1172::TaskManager.Task::(prepare) Task=`e0d2b09b-ff0f-4c33-914e-7691c27fd2ac`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-252::DEBUG::2012-12-09 12:11:06,683::task::588::TaskManager.Task::(_updateState) Task=`e0d2b09b-ff0f-4c33-914e-7691c27fd2ac`::moving from state preparing -> state finished
>Thread-252::DEBUG::2012-12-09 12:11:06,683::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-252::DEBUG::2012-12-09 12:11:06,684::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-252::DEBUG::2012-12-09 12:11:06,684::task::978::TaskManager.Task::(_decref) Task=`e0d2b09b-ff0f-4c33-914e-7691c27fd2ac`::ref 0 aborting False
>Thread-253::DEBUG::2012-12-09 12:11:07,705::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-253::DEBUG::2012-12-09 12:11:07,706::task::588::TaskManager.Task::(_updateState) Task=`fdcf26c5-d339-455c-bbbc-c0d0bdda2246`::moving from state init -> state preparing
>Thread-253::INFO::2012-12-09 12:11:07,706::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-253::DEBUG::2012-12-09 12:11:07,706::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-253::DEBUG::2012-12-09 12:11:07,706::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-253::INFO::2012-12-09 12:11:07,706::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-253::DEBUG::2012-12-09 12:11:07,707::task::1172::TaskManager.Task::(prepare) Task=`fdcf26c5-d339-455c-bbbc-c0d0bdda2246`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-253::DEBUG::2012-12-09 12:11:07,707::task::588::TaskManager.Task::(_updateState) Task=`fdcf26c5-d339-455c-bbbc-c0d0bdda2246`::moving from state preparing -> state finished
>Thread-253::DEBUG::2012-12-09 12:11:07,707::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-253::DEBUG::2012-12-09 12:11:07,707::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-253::DEBUG::2012-12-09 12:11:07,707::task::978::TaskManager.Task::(_decref) Task=`fdcf26c5-d339-455c-bbbc-c0d0bdda2246`::ref 0 aborting False
>Thread-255::DEBUG::2012-12-09 12:11:08,735::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-255::DEBUG::2012-12-09 12:11:08,736::task::588::TaskManager.Task::(_updateState) Task=`bda97bdc-84c3-4300-af5b-ba5901421f00`::moving from state init -> state preparing
>Thread-255::INFO::2012-12-09 12:11:08,736::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-255::DEBUG::2012-12-09 12:11:08,736::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-255::DEBUG::2012-12-09 12:11:08,737::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-255::INFO::2012-12-09 12:11:08,737::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-255::DEBUG::2012-12-09 12:11:08,738::task::1172::TaskManager.Task::(prepare) Task=`bda97bdc-84c3-4300-af5b-ba5901421f00`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-255::DEBUG::2012-12-09 12:11:08,738::task::588::TaskManager.Task::(_updateState) Task=`bda97bdc-84c3-4300-af5b-ba5901421f00`::moving from state preparing -> state finished
>Thread-255::DEBUG::2012-12-09 12:11:08,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-255::DEBUG::2012-12-09 12:11:08,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-255::DEBUG::2012-12-09 12:11:08,739::task::978::TaskManager.Task::(_decref) Task=`bda97bdc-84c3-4300-af5b-ba5901421f00`::ref 0 aborting False
>Thread-256::DEBUG::2012-12-09 12:11:09,762::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-256::DEBUG::2012-12-09 12:11:09,763::task::588::TaskManager.Task::(_updateState) Task=`835c401f-6a4d-4030-ae04-c4099419e1dd`::moving from state init -> state preparing
>Thread-256::INFO::2012-12-09 12:11:09,764::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-256::DEBUG::2012-12-09 12:11:09,764::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-256::DEBUG::2012-12-09 12:11:09,764::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-256::INFO::2012-12-09 12:11:09,764::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-256::DEBUG::2012-12-09 12:11:09,765::task::1172::TaskManager.Task::(prepare) Task=`835c401f-6a4d-4030-ae04-c4099419e1dd`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-256::DEBUG::2012-12-09 12:11:09,765::task::588::TaskManager.Task::(_updateState) Task=`835c401f-6a4d-4030-ae04-c4099419e1dd`::moving from state preparing -> state finished
>Thread-256::DEBUG::2012-12-09 12:11:09,765::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-256::DEBUG::2012-12-09 12:11:09,765::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-256::DEBUG::2012-12-09 12:11:09,765::task::978::TaskManager.Task::(_decref) Task=`835c401f-6a4d-4030-ae04-c4099419e1dd`::ref 0 aborting False
>Thread-258::DEBUG::2012-12-09 12:11:10,792::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-258::DEBUG::2012-12-09 12:11:10,793::task::588::TaskManager.Task::(_updateState) Task=`a0d7c33f-8a69-42b3-82bc-679b0b48078c`::moving from state init -> state preparing
>Thread-258::INFO::2012-12-09 12:11:10,793::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-258::DEBUG::2012-12-09 12:11:10,794::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-258::DEBUG::2012-12-09 12:11:10,794::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-258::INFO::2012-12-09 12:11:10,795::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-258::DEBUG::2012-12-09 12:11:10,795::task::1172::TaskManager.Task::(prepare) Task=`a0d7c33f-8a69-42b3-82bc-679b0b48078c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-258::DEBUG::2012-12-09 12:11:10,795::task::588::TaskManager.Task::(_updateState) Task=`a0d7c33f-8a69-42b3-82bc-679b0b48078c`::moving from state preparing -> state finished
>Thread-258::DEBUG::2012-12-09 12:11:10,796::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-258::DEBUG::2012-12-09 12:11:10,796::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-258::DEBUG::2012-12-09 12:11:10,797::task::978::TaskManager.Task::(_decref) Task=`a0d7c33f-8a69-42b3-82bc-679b0b48078c`::ref 0 aborting False
>Thread-259::DEBUG::2012-12-09 12:11:11,822::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-259::DEBUG::2012-12-09 12:11:11,822::task::588::TaskManager.Task::(_updateState) Task=`8295d668-b001-425c-bdfe-42d9f65be212`::moving from state init -> state preparing
>Thread-259::INFO::2012-12-09 12:11:11,823::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-259::DEBUG::2012-12-09 12:11:11,823::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-259::DEBUG::2012-12-09 12:11:11,823::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-259::INFO::2012-12-09 12:11:11,823::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-259::DEBUG::2012-12-09 12:11:11,823::task::1172::TaskManager.Task::(prepare) Task=`8295d668-b001-425c-bdfe-42d9f65be212`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-259::DEBUG::2012-12-09 12:11:11,823::task::588::TaskManager.Task::(_updateState) Task=`8295d668-b001-425c-bdfe-42d9f65be212`::moving from state preparing -> state finished
>Thread-259::DEBUG::2012-12-09 12:11:11,824::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-259::DEBUG::2012-12-09 12:11:11,824::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-259::DEBUG::2012-12-09 12:11:11,824::task::978::TaskManager.Task::(_decref) Task=`8295d668-b001-425c-bdfe-42d9f65be212`::ref 0 aborting False
>Thread-261::DEBUG::2012-12-09 12:11:12,849::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-261::DEBUG::2012-12-09 12:11:12,849::task::588::TaskManager.Task::(_updateState) Task=`741fa0aa-13fe-49a2-b438-322984c06c9b`::moving from state init -> state preparing
>Thread-261::INFO::2012-12-09 12:11:12,850::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-261::DEBUG::2012-12-09 12:11:12,850::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-261::DEBUG::2012-12-09 12:11:12,850::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-261::INFO::2012-12-09 12:11:12,850::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-261::DEBUG::2012-12-09 12:11:12,850::task::1172::TaskManager.Task::(prepare) Task=`741fa0aa-13fe-49a2-b438-322984c06c9b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-261::DEBUG::2012-12-09 12:11:12,851::task::588::TaskManager.Task::(_updateState) Task=`741fa0aa-13fe-49a2-b438-322984c06c9b`::moving from state preparing -> state finished
>Thread-261::DEBUG::2012-12-09 12:11:12,851::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-261::DEBUG::2012-12-09 12:11:12,851::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-261::DEBUG::2012-12-09 12:11:12,851::task::978::TaskManager.Task::(_decref) Task=`741fa0aa-13fe-49a2-b438-322984c06c9b`::ref 0 aborting False
>Thread-262::DEBUG::2012-12-09 12:11:13,883::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-262::DEBUG::2012-12-09 12:11:13,884::task::588::TaskManager.Task::(_updateState) Task=`7d1430b3-c54d-433d-83af-e6e97201218a`::moving from state init -> state preparing
>Thread-262::INFO::2012-12-09 12:11:13,885::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-262::DEBUG::2012-12-09 12:11:13,885::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-262::DEBUG::2012-12-09 12:11:13,885::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-262::INFO::2012-12-09 12:11:13,885::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-262::DEBUG::2012-12-09 12:11:13,886::task::1172::TaskManager.Task::(prepare) Task=`7d1430b3-c54d-433d-83af-e6e97201218a`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-262::DEBUG::2012-12-09 12:11:13,886::task::588::TaskManager.Task::(_updateState) Task=`7d1430b3-c54d-433d-83af-e6e97201218a`::moving from state preparing -> state finished
>Thread-262::DEBUG::2012-12-09 12:11:13,886::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-262::DEBUG::2012-12-09 12:11:13,887::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-262::DEBUG::2012-12-09 12:11:13,887::task::978::TaskManager.Task::(_decref) Task=`7d1430b3-c54d-433d-83af-e6e97201218a`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:14,462::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:14,463::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:11:14,579::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:14,581::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:14,582::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:14,583::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-263::DEBUG::2012-12-09 12:11:14,629::task::588::TaskManager.Task::(_updateState) Task=`3859c779-aec5-458f-8824-3528e2f58979`::moving from state init -> state preparing
>Thread-263::INFO::2012-12-09 12:11:14,629::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-263::INFO::2012-12-09 12:11:14,630::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '131.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-263::DEBUG::2012-12-09 12:11:14,630::task::1172::TaskManager.Task::(prepare) Task=`3859c779-aec5-458f-8824-3528e2f58979`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '131.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-263::DEBUG::2012-12-09 12:11:14,630::task::588::TaskManager.Task::(_updateState) Task=`3859c779-aec5-458f-8824-3528e2f58979`::moving from state preparing -> state finished
>Thread-263::DEBUG::2012-12-09 12:11:14,630::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-263::DEBUG::2012-12-09 12:11:14,630::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-263::DEBUG::2012-12-09 12:11:14,631::task::978::TaskManager.Task::(_decref) Task=`3859c779-aec5-458f-8824-3528e2f58979`::ref 0 aborting False
>Thread-265::DEBUG::2012-12-09 12:11:14,915::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-265::DEBUG::2012-12-09 12:11:14,916::task::588::TaskManager.Task::(_updateState) Task=`8762efb3-2c0b-4c8a-a7c6-b6d8cfde204b`::moving from state init -> state preparing
>Thread-265::INFO::2012-12-09 12:11:14,917::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-265::DEBUG::2012-12-09 12:11:14,917::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-265::DEBUG::2012-12-09 12:11:14,918::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-265::INFO::2012-12-09 12:11:14,918::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-265::DEBUG::2012-12-09 12:11:14,918::task::1172::TaskManager.Task::(prepare) Task=`8762efb3-2c0b-4c8a-a7c6-b6d8cfde204b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-265::DEBUG::2012-12-09 12:11:14,919::task::588::TaskManager.Task::(_updateState) Task=`8762efb3-2c0b-4c8a-a7c6-b6d8cfde204b`::moving from state preparing -> state finished
>Thread-265::DEBUG::2012-12-09 12:11:14,919::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-265::DEBUG::2012-12-09 12:11:14,920::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-265::DEBUG::2012-12-09 12:11:14,920::task::978::TaskManager.Task::(_decref) Task=`8762efb3-2c0b-4c8a-a7c6-b6d8cfde204b`::ref 0 aborting False
>Thread-266::DEBUG::2012-12-09 12:11:15,952::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-266::DEBUG::2012-12-09 12:11:15,953::task::588::TaskManager.Task::(_updateState) Task=`8e8115e6-63b2-4040-957d-5b6b0638047c`::moving from state init -> state preparing
>Thread-266::INFO::2012-12-09 12:11:15,954::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-266::DEBUG::2012-12-09 12:11:15,954::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-266::DEBUG::2012-12-09 12:11:15,954::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-266::INFO::2012-12-09 12:11:15,955::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-266::DEBUG::2012-12-09 12:11:15,955::task::1172::TaskManager.Task::(prepare) Task=`8e8115e6-63b2-4040-957d-5b6b0638047c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-266::DEBUG::2012-12-09 12:11:15,956::task::588::TaskManager.Task::(_updateState) Task=`8e8115e6-63b2-4040-957d-5b6b0638047c`::moving from state preparing -> state finished
>Thread-266::DEBUG::2012-12-09 12:11:15,956::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-266::DEBUG::2012-12-09 12:11:15,956::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-266::DEBUG::2012-12-09 12:11:15,957::task::978::TaskManager.Task::(_decref) Task=`8e8115e6-63b2-4040-957d-5b6b0638047c`::ref 0 aborting False
>Thread-268::DEBUG::2012-12-09 12:11:16,985::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-268::DEBUG::2012-12-09 12:11:16,986::task::588::TaskManager.Task::(_updateState) Task=`4d1c3ef4-accb-4bde-bf89-87958826b902`::moving from state init -> state preparing
>Thread-268::INFO::2012-12-09 12:11:16,987::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-268::DEBUG::2012-12-09 12:11:16,987::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-268::DEBUG::2012-12-09 12:11:16,987::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-268::INFO::2012-12-09 12:11:16,988::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-268::DEBUG::2012-12-09 12:11:16,988::task::1172::TaskManager.Task::(prepare) Task=`4d1c3ef4-accb-4bde-bf89-87958826b902`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-268::DEBUG::2012-12-09 12:11:16,989::task::588::TaskManager.Task::(_updateState) Task=`4d1c3ef4-accb-4bde-bf89-87958826b902`::moving from state preparing -> state finished
>Thread-268::DEBUG::2012-12-09 12:11:16,989::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-268::DEBUG::2012-12-09 12:11:16,989::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-268::DEBUG::2012-12-09 12:11:16,989::task::978::TaskManager.Task::(_decref) Task=`4d1c3ef4-accb-4bde-bf89-87958826b902`::ref 0 aborting False
>Thread-269::DEBUG::2012-12-09 12:11:18,014::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-269::DEBUG::2012-12-09 12:11:18,015::task::588::TaskManager.Task::(_updateState) Task=`336a1581-5d01-4eb1-b5b6-1ad10235351b`::moving from state init -> state preparing
>Thread-269::INFO::2012-12-09 12:11:18,015::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-269::DEBUG::2012-12-09 12:11:18,015::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-269::DEBUG::2012-12-09 12:11:18,015::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-269::INFO::2012-12-09 12:11:18,016::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-269::DEBUG::2012-12-09 12:11:18,016::task::1172::TaskManager.Task::(prepare) Task=`336a1581-5d01-4eb1-b5b6-1ad10235351b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-269::DEBUG::2012-12-09 12:11:18,016::task::588::TaskManager.Task::(_updateState) Task=`336a1581-5d01-4eb1-b5b6-1ad10235351b`::moving from state preparing -> state finished
>Thread-269::DEBUG::2012-12-09 12:11:18,016::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-269::DEBUG::2012-12-09 12:11:18,016::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-269::DEBUG::2012-12-09 12:11:18,017::task::978::TaskManager.Task::(_decref) Task=`336a1581-5d01-4eb1-b5b6-1ad10235351b`::ref 0 aborting False
>Thread-271::DEBUG::2012-12-09 12:11:19,038::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-271::DEBUG::2012-12-09 12:11:19,038::task::588::TaskManager.Task::(_updateState) Task=`45755f95-3929-42e7-8111-19543d50fd7c`::moving from state init -> state preparing
>Thread-271::INFO::2012-12-09 12:11:19,038::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-271::DEBUG::2012-12-09 12:11:19,039::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-271::DEBUG::2012-12-09 12:11:19,039::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-271::INFO::2012-12-09 12:11:19,039::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-271::DEBUG::2012-12-09 12:11:19,039::task::1172::TaskManager.Task::(prepare) Task=`45755f95-3929-42e7-8111-19543d50fd7c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-271::DEBUG::2012-12-09 12:11:19,039::task::588::TaskManager.Task::(_updateState) Task=`45755f95-3929-42e7-8111-19543d50fd7c`::moving from state preparing -> state finished
>Thread-271::DEBUG::2012-12-09 12:11:19,040::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-271::DEBUG::2012-12-09 12:11:19,040::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-271::DEBUG::2012-12-09 12:11:19,040::task::978::TaskManager.Task::(_decref) Task=`45755f95-3929-42e7-8111-19543d50fd7c`::ref 0 aborting False
>Thread-272::DEBUG::2012-12-09 12:11:20,075::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-272::DEBUG::2012-12-09 12:11:20,076::task::588::TaskManager.Task::(_updateState) Task=`8fcf5f5a-b161-4f44-b12a-fa7497636a70`::moving from state init -> state preparing
>Thread-272::INFO::2012-12-09 12:11:20,076::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-272::DEBUG::2012-12-09 12:11:20,077::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-272::DEBUG::2012-12-09 12:11:20,077::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-272::INFO::2012-12-09 12:11:20,077::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-272::DEBUG::2012-12-09 12:11:20,077::task::1172::TaskManager.Task::(prepare) Task=`8fcf5f5a-b161-4f44-b12a-fa7497636a70`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-272::DEBUG::2012-12-09 12:11:20,077::task::588::TaskManager.Task::(_updateState) Task=`8fcf5f5a-b161-4f44-b12a-fa7497636a70`::moving from state preparing -> state finished
>Thread-272::DEBUG::2012-12-09 12:11:20,077::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-272::DEBUG::2012-12-09 12:11:20,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-272::DEBUG::2012-12-09 12:11:20,078::task::978::TaskManager.Task::(_decref) Task=`8fcf5f5a-b161-4f44-b12a-fa7497636a70`::ref 0 aborting False
>Thread-274::DEBUG::2012-12-09 12:11:21,105::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-274::DEBUG::2012-12-09 12:11:21,106::task::588::TaskManager.Task::(_updateState) Task=`81238200-0129-49d7-ab01-4de06eebbb2e`::moving from state init -> state preparing
>Thread-274::INFO::2012-12-09 12:11:21,106::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-274::DEBUG::2012-12-09 12:11:21,106::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-274::DEBUG::2012-12-09 12:11:21,106::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-274::INFO::2012-12-09 12:11:21,107::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-274::DEBUG::2012-12-09 12:11:21,107::task::1172::TaskManager.Task::(prepare) Task=`81238200-0129-49d7-ab01-4de06eebbb2e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-274::DEBUG::2012-12-09 12:11:21,107::task::588::TaskManager.Task::(_updateState) Task=`81238200-0129-49d7-ab01-4de06eebbb2e`::moving from state preparing -> state finished
>Thread-274::DEBUG::2012-12-09 12:11:21,107::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-274::DEBUG::2012-12-09 12:11:21,108::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-274::DEBUG::2012-12-09 12:11:21,108::task::978::TaskManager.Task::(_decref) Task=`81238200-0129-49d7-ab01-4de06eebbb2e`::ref 0 aborting False
>Thread-275::DEBUG::2012-12-09 12:11:22,139::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-275::DEBUG::2012-12-09 12:11:22,140::task::588::TaskManager.Task::(_updateState) Task=`db141e72-2163-41dd-926f-6c38df82111d`::moving from state init -> state preparing
>Thread-275::INFO::2012-12-09 12:11:22,141::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-275::DEBUG::2012-12-09 12:11:22,141::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-275::DEBUG::2012-12-09 12:11:22,141::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-275::INFO::2012-12-09 12:11:22,142::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-275::DEBUG::2012-12-09 12:11:22,142::task::1172::TaskManager.Task::(prepare) Task=`db141e72-2163-41dd-926f-6c38df82111d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-275::DEBUG::2012-12-09 12:11:22,143::task::588::TaskManager.Task::(_updateState) Task=`db141e72-2163-41dd-926f-6c38df82111d`::moving from state preparing -> state finished
>Thread-275::DEBUG::2012-12-09 12:11:22,143::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-275::DEBUG::2012-12-09 12:11:22,143::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-275::DEBUG::2012-12-09 12:11:22,144::task::978::TaskManager.Task::(_decref) Task=`db141e72-2163-41dd-926f-6c38df82111d`::ref 0 aborting False
>Thread-277::DEBUG::2012-12-09 12:11:23,166::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-277::DEBUG::2012-12-09 12:11:23,167::task::588::TaskManager.Task::(_updateState) Task=`ac37af53-6075-4143-8e77-b508921ae6f7`::moving from state init -> state preparing
>Thread-277::INFO::2012-12-09 12:11:23,167::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-277::DEBUG::2012-12-09 12:11:23,167::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-277::DEBUG::2012-12-09 12:11:23,167::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-277::INFO::2012-12-09 12:11:23,168::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-277::DEBUG::2012-12-09 12:11:23,168::task::1172::TaskManager.Task::(prepare) Task=`ac37af53-6075-4143-8e77-b508921ae6f7`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-277::DEBUG::2012-12-09 12:11:23,168::task::588::TaskManager.Task::(_updateState) Task=`ac37af53-6075-4143-8e77-b508921ae6f7`::moving from state preparing -> state finished
>Thread-277::DEBUG::2012-12-09 12:11:23,168::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-277::DEBUG::2012-12-09 12:11:23,169::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-277::DEBUG::2012-12-09 12:11:23,169::task::978::TaskManager.Task::(_decref) Task=`ac37af53-6075-4143-8e77-b508921ae6f7`::ref 0 aborting False
>Thread-278::DEBUG::2012-12-09 12:11:24,191::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-278::DEBUG::2012-12-09 12:11:24,192::task::588::TaskManager.Task::(_updateState) Task=`f417df54-6349-4324-9fe6-5406d012e57b`::moving from state init -> state preparing
>Thread-278::INFO::2012-12-09 12:11:24,192::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-278::DEBUG::2012-12-09 12:11:24,192::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-278::DEBUG::2012-12-09 12:11:24,192::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-278::INFO::2012-12-09 12:11:24,192::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-278::DEBUG::2012-12-09 12:11:24,193::task::1172::TaskManager.Task::(prepare) Task=`f417df54-6349-4324-9fe6-5406d012e57b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-278::DEBUG::2012-12-09 12:11:24,193::task::588::TaskManager.Task::(_updateState) Task=`f417df54-6349-4324-9fe6-5406d012e57b`::moving from state preparing -> state finished
>Thread-278::DEBUG::2012-12-09 12:11:24,193::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-278::DEBUG::2012-12-09 12:11:24,193::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-278::DEBUG::2012-12-09 12:11:24,193::task::978::TaskManager.Task::(_decref) Task=`f417df54-6349-4324-9fe6-5406d012e57b`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:24,584::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:24,584::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:11:24,700::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:24,701::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:24,701::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:24,702::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-279::DEBUG::2012-12-09 12:11:24,875::task::588::TaskManager.Task::(_updateState) Task=`5ddeb3bc-e6fc-4b70-af85-dc85012441e8`::moving from state init -> state preparing
>Thread-279::INFO::2012-12-09 12:11:24,876::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-279::INFO::2012-12-09 12:11:24,876::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '141.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-279::DEBUG::2012-12-09 12:11:24,877::task::1172::TaskManager.Task::(prepare) Task=`5ddeb3bc-e6fc-4b70-af85-dc85012441e8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '141.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-279::DEBUG::2012-12-09 12:11:24,877::task::588::TaskManager.Task::(_updateState) Task=`5ddeb3bc-e6fc-4b70-af85-dc85012441e8`::moving from state preparing -> state finished
>Thread-279::DEBUG::2012-12-09 12:11:24,878::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-279::DEBUG::2012-12-09 12:11:24,878::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-279::DEBUG::2012-12-09 12:11:24,879::task::978::TaskManager.Task::(_decref) Task=`5ddeb3bc-e6fc-4b70-af85-dc85012441e8`::ref 0 aborting False
>Thread-281::DEBUG::2012-12-09 12:11:25,225::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-281::DEBUG::2012-12-09 12:11:25,226::task::588::TaskManager.Task::(_updateState) Task=`8bb226ba-cb48-4ee5-82a5-f84fe6e187ee`::moving from state init -> state preparing
>Thread-281::INFO::2012-12-09 12:11:25,226::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-281::DEBUG::2012-12-09 12:11:25,226::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-281::DEBUG::2012-12-09 12:11:25,227::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-281::INFO::2012-12-09 12:11:25,227::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-281::DEBUG::2012-12-09 12:11:25,227::task::1172::TaskManager.Task::(prepare) Task=`8bb226ba-cb48-4ee5-82a5-f84fe6e187ee`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-281::DEBUG::2012-12-09 12:11:25,227::task::588::TaskManager.Task::(_updateState) Task=`8bb226ba-cb48-4ee5-82a5-f84fe6e187ee`::moving from state preparing -> state finished
>Thread-281::DEBUG::2012-12-09 12:11:25,228::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-281::DEBUG::2012-12-09 12:11:25,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-281::DEBUG::2012-12-09 12:11:25,228::task::978::TaskManager.Task::(_decref) Task=`8bb226ba-cb48-4ee5-82a5-f84fe6e187ee`::ref 0 aborting False
>Thread-282::DEBUG::2012-12-09 12:11:26,264::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-282::DEBUG::2012-12-09 12:11:26,265::task::588::TaskManager.Task::(_updateState) Task=`750bab60-b856-401a-98cd-7a547235f001`::moving from state init -> state preparing
>Thread-282::INFO::2012-12-09 12:11:26,266::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-282::DEBUG::2012-12-09 12:11:26,266::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-282::DEBUG::2012-12-09 12:11:26,266::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-282::INFO::2012-12-09 12:11:26,267::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-282::DEBUG::2012-12-09 12:11:26,267::task::1172::TaskManager.Task::(prepare) Task=`750bab60-b856-401a-98cd-7a547235f001`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-282::DEBUG::2012-12-09 12:11:26,268::task::588::TaskManager.Task::(_updateState) Task=`750bab60-b856-401a-98cd-7a547235f001`::moving from state preparing -> state finished
>Thread-282::DEBUG::2012-12-09 12:11:26,268::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-282::DEBUG::2012-12-09 12:11:26,268::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-282::DEBUG::2012-12-09 12:11:26,269::task::978::TaskManager.Task::(_decref) Task=`750bab60-b856-401a-98cd-7a547235f001`::ref 0 aborting False
>Thread-284::DEBUG::2012-12-09 12:11:27,290::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-284::DEBUG::2012-12-09 12:11:27,291::task::588::TaskManager.Task::(_updateState) Task=`922265ae-8938-4534-8356-2128ed68d315`::moving from state init -> state preparing
>Thread-284::INFO::2012-12-09 12:11:27,291::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-284::DEBUG::2012-12-09 12:11:27,291::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-284::DEBUG::2012-12-09 12:11:27,292::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-284::INFO::2012-12-09 12:11:27,292::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-284::DEBUG::2012-12-09 12:11:27,292::task::1172::TaskManager.Task::(prepare) Task=`922265ae-8938-4534-8356-2128ed68d315`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-284::DEBUG::2012-12-09 12:11:27,292::task::588::TaskManager.Task::(_updateState) Task=`922265ae-8938-4534-8356-2128ed68d315`::moving from state preparing -> state finished
>Thread-284::DEBUG::2012-12-09 12:11:27,292::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-284::DEBUG::2012-12-09 12:11:27,292::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-284::DEBUG::2012-12-09 12:11:27,293::task::978::TaskManager.Task::(_decref) Task=`922265ae-8938-4534-8356-2128ed68d315`::ref 0 aborting False
>Thread-285::DEBUG::2012-12-09 12:11:28,319::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-285::DEBUG::2012-12-09 12:11:28,320::task::588::TaskManager.Task::(_updateState) Task=`d962a5a2-1e37-4514-9a9d-28050ac26729`::moving from state init -> state preparing
>Thread-285::INFO::2012-12-09 12:11:28,320::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-285::DEBUG::2012-12-09 12:11:28,320::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-285::DEBUG::2012-12-09 12:11:28,320::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-285::INFO::2012-12-09 12:11:28,320::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-285::DEBUG::2012-12-09 12:11:28,320::task::1172::TaskManager.Task::(prepare) Task=`d962a5a2-1e37-4514-9a9d-28050ac26729`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-285::DEBUG::2012-12-09 12:11:28,321::task::588::TaskManager.Task::(_updateState) Task=`d962a5a2-1e37-4514-9a9d-28050ac26729`::moving from state preparing -> state finished
>Thread-285::DEBUG::2012-12-09 12:11:28,321::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-285::DEBUG::2012-12-09 12:11:28,321::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-285::DEBUG::2012-12-09 12:11:28,321::task::978::TaskManager.Task::(_decref) Task=`d962a5a2-1e37-4514-9a9d-28050ac26729`::ref 0 aborting False
>Thread-287::DEBUG::2012-12-09 12:11:29,354::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-287::DEBUG::2012-12-09 12:11:29,355::task::588::TaskManager.Task::(_updateState) Task=`0c736e30-1b83-4510-8c07-ceb396701ee4`::moving from state init -> state preparing
>Thread-287::INFO::2012-12-09 12:11:29,355::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-287::DEBUG::2012-12-09 12:11:29,356::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-287::DEBUG::2012-12-09 12:11:29,356::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-287::INFO::2012-12-09 12:11:29,357::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-287::DEBUG::2012-12-09 12:11:29,357::task::1172::TaskManager.Task::(prepare) Task=`0c736e30-1b83-4510-8c07-ceb396701ee4`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-287::DEBUG::2012-12-09 12:11:29,357::task::588::TaskManager.Task::(_updateState) Task=`0c736e30-1b83-4510-8c07-ceb396701ee4`::moving from state preparing -> state finished
>Thread-287::DEBUG::2012-12-09 12:11:29,358::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-287::DEBUG::2012-12-09 12:11:29,358::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-287::DEBUG::2012-12-09 12:11:29,359::task::978::TaskManager.Task::(_decref) Task=`0c736e30-1b83-4510-8c07-ceb396701ee4`::ref 0 aborting False
>Thread-288::DEBUG::2012-12-09 12:11:30,393::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-288::DEBUG::2012-12-09 12:11:30,394::task::588::TaskManager.Task::(_updateState) Task=`e8b06c24-97eb-4008-b001-391477f7004f`::moving from state init -> state preparing
>Thread-288::INFO::2012-12-09 12:11:30,394::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-288::DEBUG::2012-12-09 12:11:30,394::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-288::DEBUG::2012-12-09 12:11:30,395::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-288::INFO::2012-12-09 12:11:30,395::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-288::DEBUG::2012-12-09 12:11:30,396::task::1172::TaskManager.Task::(prepare) Task=`e8b06c24-97eb-4008-b001-391477f7004f`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-288::DEBUG::2012-12-09 12:11:30,396::task::588::TaskManager.Task::(_updateState) Task=`e8b06c24-97eb-4008-b001-391477f7004f`::moving from state preparing -> state finished
>Thread-288::DEBUG::2012-12-09 12:11:30,397::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-288::DEBUG::2012-12-09 12:11:30,397::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-288::DEBUG::2012-12-09 12:11:30,397::task::978::TaskManager.Task::(_decref) Task=`e8b06c24-97eb-4008-b001-391477f7004f`::ref 0 aborting False
>Thread-290::DEBUG::2012-12-09 12:11:31,431::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-290::DEBUG::2012-12-09 12:11:31,432::task::588::TaskManager.Task::(_updateState) Task=`acf53601-470c-4292-bc6e-dc23fa0a4852`::moving from state init -> state preparing
>Thread-290::INFO::2012-12-09 12:11:31,433::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-290::DEBUG::2012-12-09 12:11:31,433::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-290::DEBUG::2012-12-09 12:11:31,434::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-290::INFO::2012-12-09 12:11:31,434::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-290::DEBUG::2012-12-09 12:11:31,434::task::1172::TaskManager.Task::(prepare) Task=`acf53601-470c-4292-bc6e-dc23fa0a4852`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-290::DEBUG::2012-12-09 12:11:31,435::task::588::TaskManager.Task::(_updateState) Task=`acf53601-470c-4292-bc6e-dc23fa0a4852`::moving from state preparing -> state finished
>Thread-290::DEBUG::2012-12-09 12:11:31,435::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-290::DEBUG::2012-12-09 12:11:31,436::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-290::DEBUG::2012-12-09 12:11:31,436::task::978::TaskManager.Task::(_decref) Task=`acf53601-470c-4292-bc6e-dc23fa0a4852`::ref 0 aborting False
>Thread-291::DEBUG::2012-12-09 12:11:32,474::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-291::DEBUG::2012-12-09 12:11:32,475::task::588::TaskManager.Task::(_updateState) Task=`84f37292-37f7-4223-822d-1bb0fd6a5e7d`::moving from state init -> state preparing
>Thread-291::INFO::2012-12-09 12:11:32,475::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-291::DEBUG::2012-12-09 12:11:32,476::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-291::DEBUG::2012-12-09 12:11:32,476::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-291::INFO::2012-12-09 12:11:32,476::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-291::DEBUG::2012-12-09 12:11:32,476::task::1172::TaskManager.Task::(prepare) Task=`84f37292-37f7-4223-822d-1bb0fd6a5e7d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-291::DEBUG::2012-12-09 12:11:32,477::task::588::TaskManager.Task::(_updateState) Task=`84f37292-37f7-4223-822d-1bb0fd6a5e7d`::moving from state preparing -> state finished
>Thread-291::DEBUG::2012-12-09 12:11:32,477::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-291::DEBUG::2012-12-09 12:11:32,477::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-291::DEBUG::2012-12-09 12:11:32,478::task::978::TaskManager.Task::(_decref) Task=`84f37292-37f7-4223-822d-1bb0fd6a5e7d`::ref 0 aborting False
>Thread-293::DEBUG::2012-12-09 12:11:33,515::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-293::DEBUG::2012-12-09 12:11:33,516::task::588::TaskManager.Task::(_updateState) Task=`4235833a-55d6-4e5d-a205-1f30468b7e0c`::moving from state init -> state preparing
>Thread-293::INFO::2012-12-09 12:11:33,516::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-293::DEBUG::2012-12-09 12:11:33,517::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-293::DEBUG::2012-12-09 12:11:33,517::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-293::INFO::2012-12-09 12:11:33,518::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-293::DEBUG::2012-12-09 12:11:33,518::task::1172::TaskManager.Task::(prepare) Task=`4235833a-55d6-4e5d-a205-1f30468b7e0c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-293::DEBUG::2012-12-09 12:11:33,518::task::588::TaskManager.Task::(_updateState) Task=`4235833a-55d6-4e5d-a205-1f30468b7e0c`::moving from state preparing -> state finished
>Thread-293::DEBUG::2012-12-09 12:11:33,519::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-293::DEBUG::2012-12-09 12:11:33,519::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-293::DEBUG::2012-12-09 12:11:33,519::task::978::TaskManager.Task::(_decref) Task=`4235833a-55d6-4e5d-a205-1f30468b7e0c`::ref 0 aborting False
>Thread-294::DEBUG::2012-12-09 12:11:34,542::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-294::DEBUG::2012-12-09 12:11:34,542::task::588::TaskManager.Task::(_updateState) Task=`570aa6ff-dbea-4c35-a917-baabfc3e229c`::moving from state init -> state preparing
>Thread-294::INFO::2012-12-09 12:11:34,543::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-294::DEBUG::2012-12-09 12:11:34,543::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-294::DEBUG::2012-12-09 12:11:34,543::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-294::INFO::2012-12-09 12:11:34,543::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-294::DEBUG::2012-12-09 12:11:34,543::task::1172::TaskManager.Task::(prepare) Task=`570aa6ff-dbea-4c35-a917-baabfc3e229c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-294::DEBUG::2012-12-09 12:11:34,544::task::588::TaskManager.Task::(_updateState) Task=`570aa6ff-dbea-4c35-a917-baabfc3e229c`::moving from state preparing -> state finished
>Thread-294::DEBUG::2012-12-09 12:11:34,544::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-294::DEBUG::2012-12-09 12:11:34,544::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-294::DEBUG::2012-12-09 12:11:34,544::task::978::TaskManager.Task::(_decref) Task=`570aa6ff-dbea-4c35-a917-baabfc3e229c`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:34,702::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:34,703::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:11:34,814::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:34,816::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:34,816::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:34,817::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-295::DEBUG::2012-12-09 12:11:35,138::task::588::TaskManager.Task::(_updateState) Task=`1d7e188a-c15f-4131-ada9-8ed731d3157d`::moving from state init -> state preparing
>Thread-295::INFO::2012-12-09 12:11:35,139::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-295::INFO::2012-12-09 12:11:35,139::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '152.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-295::DEBUG::2012-12-09 12:11:35,140::task::1172::TaskManager.Task::(prepare) Task=`1d7e188a-c15f-4131-ada9-8ed731d3157d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0217809677124', 'lastCheck': '152.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-295::DEBUG::2012-12-09 12:11:35,140::task::588::TaskManager.Task::(_updateState) Task=`1d7e188a-c15f-4131-ada9-8ed731d3157d`::moving from state preparing -> state finished
>Thread-295::DEBUG::2012-12-09 12:11:35,141::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-295::DEBUG::2012-12-09 12:11:35,141::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-295::DEBUG::2012-12-09 12:11:35,141::task::978::TaskManager.Task::(_decref) Task=`1d7e188a-c15f-4131-ada9-8ed731d3157d`::ref 0 aborting False
>Thread-297::DEBUG::2012-12-09 12:11:35,566::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-297::DEBUG::2012-12-09 12:11:35,566::task::588::TaskManager.Task::(_updateState) Task=`b98f6a77-3bba-4c88-8891-a6a6a1110f6c`::moving from state init -> state preparing
>Thread-297::INFO::2012-12-09 12:11:35,566::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-297::DEBUG::2012-12-09 12:11:35,567::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-297::DEBUG::2012-12-09 12:11:35,567::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-297::INFO::2012-12-09 12:11:35,567::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-297::DEBUG::2012-12-09 12:11:35,567::task::1172::TaskManager.Task::(prepare) Task=`b98f6a77-3bba-4c88-8891-a6a6a1110f6c`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-297::DEBUG::2012-12-09 12:11:35,567::task::588::TaskManager.Task::(_updateState) Task=`b98f6a77-3bba-4c88-8891-a6a6a1110f6c`::moving from state preparing -> state finished
>Thread-297::DEBUG::2012-12-09 12:11:35,568::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-297::DEBUG::2012-12-09 12:11:35,568::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-297::DEBUG::2012-12-09 12:11:35,568::task::978::TaskManager.Task::(_decref) Task=`b98f6a77-3bba-4c88-8891-a6a6a1110f6c`::ref 0 aborting False
>Thread-298::DEBUG::2012-12-09 12:11:36,602::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-298::DEBUG::2012-12-09 12:11:36,603::task::588::TaskManager.Task::(_updateState) Task=`7b18107c-90fc-4898-94f6-ffd794ca14ff`::moving from state init -> state preparing
>Thread-298::INFO::2012-12-09 12:11:36,603::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-298::DEBUG::2012-12-09 12:11:36,603::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-298::DEBUG::2012-12-09 12:11:36,603::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-298::INFO::2012-12-09 12:11:36,603::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-298::DEBUG::2012-12-09 12:11:36,604::task::1172::TaskManager.Task::(prepare) Task=`7b18107c-90fc-4898-94f6-ffd794ca14ff`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-298::DEBUG::2012-12-09 12:11:36,604::task::588::TaskManager.Task::(_updateState) Task=`7b18107c-90fc-4898-94f6-ffd794ca14ff`::moving from state preparing -> state finished
>Thread-298::DEBUG::2012-12-09 12:11:36,604::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-298::DEBUG::2012-12-09 12:11:36,604::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-298::DEBUG::2012-12-09 12:11:36,604::task::978::TaskManager.Task::(_decref) Task=`7b18107c-90fc-4898-94f6-ffd794ca14ff`::ref 0 aborting False
>Thread-300::DEBUG::2012-12-09 12:11:37,637::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-300::DEBUG::2012-12-09 12:11:37,638::task::588::TaskManager.Task::(_updateState) Task=`5b19cd0e-1bd8-46d9-9560-42eb4a9c553b`::moving from state init -> state preparing
>Thread-300::INFO::2012-12-09 12:11:37,638::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-300::DEBUG::2012-12-09 12:11:37,638::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-300::DEBUG::2012-12-09 12:11:37,639::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-300::INFO::2012-12-09 12:11:37,639::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-300::DEBUG::2012-12-09 12:11:37,640::task::1172::TaskManager.Task::(prepare) Task=`5b19cd0e-1bd8-46d9-9560-42eb4a9c553b`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-300::DEBUG::2012-12-09 12:11:37,640::task::588::TaskManager.Task::(_updateState) Task=`5b19cd0e-1bd8-46d9-9560-42eb4a9c553b`::moving from state preparing -> state finished
>Thread-300::DEBUG::2012-12-09 12:11:37,640::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-300::DEBUG::2012-12-09 12:11:37,641::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-300::DEBUG::2012-12-09 12:11:37,641::task::978::TaskManager.Task::(_decref) Task=`5b19cd0e-1bd8-46d9-9560-42eb4a9c553b`::ref 0 aborting False
>Thread-301::DEBUG::2012-12-09 12:11:38,663::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-301::DEBUG::2012-12-09 12:11:38,663::task::588::TaskManager.Task::(_updateState) Task=`9f8b0799-8c9c-4005-8cc8-114c44e91b87`::moving from state init -> state preparing
>Thread-301::INFO::2012-12-09 12:11:38,664::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-301::DEBUG::2012-12-09 12:11:38,664::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-301::DEBUG::2012-12-09 12:11:38,664::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-301::INFO::2012-12-09 12:11:38,664::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-301::DEBUG::2012-12-09 12:11:38,664::task::1172::TaskManager.Task::(prepare) Task=`9f8b0799-8c9c-4005-8cc8-114c44e91b87`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-301::DEBUG::2012-12-09 12:11:38,665::task::588::TaskManager.Task::(_updateState) Task=`9f8b0799-8c9c-4005-8cc8-114c44e91b87`::moving from state preparing -> state finished
>Thread-301::DEBUG::2012-12-09 12:11:38,665::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-301::DEBUG::2012-12-09 12:11:38,665::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-301::DEBUG::2012-12-09 12:11:38,665::task::978::TaskManager.Task::(_decref) Task=`9f8b0799-8c9c-4005-8cc8-114c44e91b87`::ref 0 aborting False
>Thread-303::DEBUG::2012-12-09 12:11:39,687::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-303::DEBUG::2012-12-09 12:11:39,688::task::588::TaskManager.Task::(_updateState) Task=`2fb33dce-1a44-4e48-b505-67d383d12e34`::moving from state init -> state preparing
>Thread-303::INFO::2012-12-09 12:11:39,689::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-303::DEBUG::2012-12-09 12:11:39,689::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-303::DEBUG::2012-12-09 12:11:39,689::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-303::INFO::2012-12-09 12:11:39,690::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-303::DEBUG::2012-12-09 12:11:39,690::task::1172::TaskManager.Task::(prepare) Task=`2fb33dce-1a44-4e48-b505-67d383d12e34`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-303::DEBUG::2012-12-09 12:11:39,690::task::588::TaskManager.Task::(_updateState) Task=`2fb33dce-1a44-4e48-b505-67d383d12e34`::moving from state preparing -> state finished
>Thread-303::DEBUG::2012-12-09 12:11:39,690::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-303::DEBUG::2012-12-09 12:11:39,690::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-303::DEBUG::2012-12-09 12:11:39,690::task::978::TaskManager.Task::(_decref) Task=`2fb33dce-1a44-4e48-b505-67d383d12e34`::ref 0 aborting False
>Thread-304::DEBUG::2012-12-09 12:11:40,712::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-304::DEBUG::2012-12-09 12:11:40,712::task::588::TaskManager.Task::(_updateState) Task=`9f65addf-9ffa-4a9a-9e4b-3990fde7173d`::moving from state init -> state preparing
>Thread-304::INFO::2012-12-09 12:11:40,713::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-304::DEBUG::2012-12-09 12:11:40,713::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-304::DEBUG::2012-12-09 12:11:40,713::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-304::INFO::2012-12-09 12:11:40,713::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-304::DEBUG::2012-12-09 12:11:40,713::task::1172::TaskManager.Task::(prepare) Task=`9f65addf-9ffa-4a9a-9e4b-3990fde7173d`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-304::DEBUG::2012-12-09 12:11:40,714::task::588::TaskManager.Task::(_updateState) Task=`9f65addf-9ffa-4a9a-9e4b-3990fde7173d`::moving from state preparing -> state finished
>Thread-304::DEBUG::2012-12-09 12:11:40,714::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-304::DEBUG::2012-12-09 12:11:40,714::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-304::DEBUG::2012-12-09 12:11:40,714::task::978::TaskManager.Task::(_decref) Task=`9f65addf-9ffa-4a9a-9e4b-3990fde7173d`::ref 0 aborting False
>Thread-306::DEBUG::2012-12-09 12:11:41,736::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-306::DEBUG::2012-12-09 12:11:41,736::task::588::TaskManager.Task::(_updateState) Task=`5e560511-423e-4793-aafb-430a6376b269`::moving from state init -> state preparing
>Thread-306::INFO::2012-12-09 12:11:41,737::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-306::DEBUG::2012-12-09 12:11:41,737::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-306::DEBUG::2012-12-09 12:11:41,738::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-306::INFO::2012-12-09 12:11:41,738::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-306::DEBUG::2012-12-09 12:11:41,738::task::1172::TaskManager.Task::(prepare) Task=`5e560511-423e-4793-aafb-430a6376b269`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-306::DEBUG::2012-12-09 12:11:41,739::task::588::TaskManager.Task::(_updateState) Task=`5e560511-423e-4793-aafb-430a6376b269`::moving from state preparing -> state finished
>Thread-306::DEBUG::2012-12-09 12:11:41,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-306::DEBUG::2012-12-09 12:11:41,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-306::DEBUG::2012-12-09 12:11:41,739::task::978::TaskManager.Task::(_decref) Task=`5e560511-423e-4793-aafb-430a6376b269`::ref 0 aborting False
>Thread-307::DEBUG::2012-12-09 12:11:42,762::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-307::DEBUG::2012-12-09 12:11:42,763::task::588::TaskManager.Task::(_updateState) Task=`476ad5ca-d8de-4d79-870d-5e7b2ef2260e`::moving from state init -> state preparing
>Thread-307::INFO::2012-12-09 12:11:42,763::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-307::DEBUG::2012-12-09 12:11:42,763::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-307::DEBUG::2012-12-09 12:11:42,763::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-307::INFO::2012-12-09 12:11:42,763::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-307::DEBUG::2012-12-09 12:11:42,764::task::1172::TaskManager.Task::(prepare) Task=`476ad5ca-d8de-4d79-870d-5e7b2ef2260e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-307::DEBUG::2012-12-09 12:11:42,764::task::588::TaskManager.Task::(_updateState) Task=`476ad5ca-d8de-4d79-870d-5e7b2ef2260e`::moving from state preparing -> state finished
>Thread-307::DEBUG::2012-12-09 12:11:42,764::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-307::DEBUG::2012-12-09 12:11:42,764::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-307::DEBUG::2012-12-09 12:11:42,764::task::978::TaskManager.Task::(_decref) Task=`476ad5ca-d8de-4d79-870d-5e7b2ef2260e`::ref 0 aborting False
>Thread-309::DEBUG::2012-12-09 12:11:43,787::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-309::DEBUG::2012-12-09 12:11:43,787::task::588::TaskManager.Task::(_updateState) Task=`95a5ab9f-8596-44ae-89d3-d47c71dd10d3`::moving from state init -> state preparing
>Thread-309::INFO::2012-12-09 12:11:43,788::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-309::DEBUG::2012-12-09 12:11:43,788::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-309::DEBUG::2012-12-09 12:11:43,788::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-309::INFO::2012-12-09 12:11:43,788::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-309::DEBUG::2012-12-09 12:11:43,788::task::1172::TaskManager.Task::(prepare) Task=`95a5ab9f-8596-44ae-89d3-d47c71dd10d3`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-309::DEBUG::2012-12-09 12:11:43,788::task::588::TaskManager.Task::(_updateState) Task=`95a5ab9f-8596-44ae-89d3-d47c71dd10d3`::moving from state preparing -> state finished
>Thread-309::DEBUG::2012-12-09 12:11:43,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-309::DEBUG::2012-12-09 12:11:43,789::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-309::DEBUG::2012-12-09 12:11:43,789::task::978::TaskManager.Task::(_decref) Task=`95a5ab9f-8596-44ae-89d3-d47c71dd10d3`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:11:44,819::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:44,819::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-310::DEBUG::2012-12-09 12:11:44,825::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-310::DEBUG::2012-12-09 12:11:44,826::task::588::TaskManager.Task::(_updateState) Task=`1408137b-ed47-4f7e-acf7-11272cda2da9`::moving from state init -> state preparing
>Thread-310::INFO::2012-12-09 12:11:44,826::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-310::DEBUG::2012-12-09 12:11:44,827::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-310::DEBUG::2012-12-09 12:11:44,827::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-310::INFO::2012-12-09 12:11:44,828::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-310::DEBUG::2012-12-09 12:11:44,828::task::1172::TaskManager.Task::(prepare) Task=`1408137b-ed47-4f7e-acf7-11272cda2da9`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-310::DEBUG::2012-12-09 12:11:44,828::task::588::TaskManager.Task::(_updateState) Task=`1408137b-ed47-4f7e-acf7-11272cda2da9`::moving from state preparing -> state finished
>Thread-310::DEBUG::2012-12-09 12:11:44,829::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-310::DEBUG::2012-12-09 12:11:44,829::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-310::DEBUG::2012-12-09 12:11:44,829::task::978::TaskManager.Task::(_decref) Task=`1408137b-ed47-4f7e-acf7-11272cda2da9`::ref 0 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,901::safelease::178::SANLock::(acquireHostId) Host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 successfully acquired (id: 2)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:44,902::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/images already exists
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:44,902::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/dom_md already exists
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,903::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-51::DEBUG::2012-12-09 12:11:44,937::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:44,939::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:44,939::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,940::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:44,941::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,942::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,943::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,943::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,944::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,944::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,953::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000655098 s, 3.1 MB/s\n'; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,954::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000655098 s, 3.1 MB/s'], size: 2048
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,954::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:44,955::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,955::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,956::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,956::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,957::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,957::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,958::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:44,959::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,063::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,064::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,065::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::INFO::2012-12-09 12:11:45,065::safelease::211::SANLock::(acquire) Acquiring cluster lock for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (id: 2)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,253::safelease::235::SANLock::(acquire) Cluster lock for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 successfully acquired (id: 2)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,254::sp::263::Storage.StoragePool::(startSpm) spm lock acquired successfully
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,254::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,255::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,255::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,256::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,256::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,257::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,368::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,370::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,370::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=48', '_SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,370::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,371::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,371::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=49', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'ROLE=Master', 'SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,371::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,371::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,371::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,372::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,372::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,372::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-311::DEBUG::2012-12-09 12:11:45,425::task::588::TaskManager.Task::(_updateState) Task=`be229429-e6a0-4a19-810c-2a44aa302f70`::moving from state init -> state preparing
>Thread-311::INFO::2012-12-09 12:11:45,425::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-311::INFO::2012-12-09 12:11:45,426::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0244600772858', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-311::DEBUG::2012-12-09 12:11:45,426::task::1172::TaskManager.Task::(prepare) Task=`be229429-e6a0-4a19-810c-2a44aa302f70`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0244600772858', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-311::DEBUG::2012-12-09 12:11:45,426::task::588::TaskManager.Task::(_updateState) Task=`be229429-e6a0-4a19-810c-2a44aa302f70`::moving from state preparing -> state finished
>Thread-311::DEBUG::2012-12-09 12:11:45,426::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-311::DEBUG::2012-12-09 12:11:45,426::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-311::DEBUG::2012-12-09 12:11:45,426::task::978::TaskManager.Task::(_decref) Task=`be229429-e6a0-4a19-810c-2a44aa302f70`::ref 0 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,472::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,474::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,474::blockSD::297::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_LVER=49, MDT__SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850 removing=MDT__SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191, MDT_POOL_SPM_LVER=48
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,475::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_LVER=48 --deltag MDT__SHA_CKSUM=eef9303a78f5cd81fd3f9fd1d6f7f7aa9399a191 --addtag MDT_POOL_SPM_LVER=49 --addtag MDT__SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850 c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,844::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,845::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,845::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,846::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,848::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`010f6791-20a0-4d78-b01c-b31a746ea2a0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,848::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,849::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,849::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`010f6791-20a0-4d78-b01c-b31a746ea2a0`::Granted request
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::INFO::2012-12-09 12:11:45,850::sp::412::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,850::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`9c64bc76-b67a-49bb-a120-098f7188da3b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,850::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' for lock type 'exclusive'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,850::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free. Now locking as 'exclusive' (1 active user)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,850::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`9c64bc76-b67a-49bb-a120-098f7188da3b`::Granted request
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,851::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,851::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-313::DEBUG::2012-12-09 12:11:45,866::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-313::DEBUG::2012-12-09 12:11:45,866::task::588::TaskManager.Task::(_updateState) Task=`7bce9aa9-b05a-43cb-95a5-3f132fdc1bbb`::moving from state init -> state preparing
>Thread-313::INFO::2012-12-09 12:11:45,867::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-313::DEBUG::2012-12-09 12:11:45,867::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-313::DEBUG::2012-12-09 12:11:45,867::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-313::INFO::2012-12-09 12:11:45,867::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-313::DEBUG::2012-12-09 12:11:45,867::task::1172::TaskManager.Task::(prepare) Task=`7bce9aa9-b05a-43cb-95a5-3f132fdc1bbb`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-313::DEBUG::2012-12-09 12:11:45,867::task::588::TaskManager.Task::(_updateState) Task=`7bce9aa9-b05a-43cb-95a5-3f132fdc1bbb`::moving from state preparing -> state finished
>Thread-313::DEBUG::2012-12-09 12:11:45,868::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-313::DEBUG::2012-12-09 12:11:45,868::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-313::DEBUG::2012-12-09 12:11:45,868::task::978::TaskManager.Task::(_decref) Task=`7bce9aa9-b05a-43cb-95a5-3f132fdc1bbb`::ref 0 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,954::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,956::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,957::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,957::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,969::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000681648 s, 3.0 MB/s\n'; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,970::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000681648 s, 3.0 MB/s'], size: 2048
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,970::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:45,970::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,970::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,970::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,971::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,971::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,971::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,972::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:45,972::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,076::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,078::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,079::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:46,080::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:46,080::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:46,080::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,080::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,081::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (0 active users)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,081::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free, finding out if anyone is waiting for it.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,081::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', Clearing records.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,081::sp::416::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,081::sp::423::Storage.StoragePool::(_upgradePool) Registering with state change event
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,082::sp::425::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads
>Thread-314::DEBUG::2012-12-09 12:11:46,082::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,083::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-314::DEBUG::2012-12-09 12:11:46,084::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,084::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,092::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,092::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,093::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/fsck -p /dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>Thread-314::DEBUG::2012-12-09 12:11:46,187::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-314::WARNING::2012-12-09 12:11:46,188::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-314::DEBUG::2012-12-09 12:11:46,188::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,445::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,446::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,484::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,485::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/mount -t ext3 /dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,798::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,815::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,875::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,876::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-315::DEBUG::2012-12-09 12:11:46,897::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-315::DEBUG::2012-12-09 12:11:46,901::task::588::TaskManager.Task::(_updateState) Task=`649d2218-a14e-406a-994a-624274aeb06e`::moving from state init -> state preparing
>Thread-315::INFO::2012-12-09 12:11:46,901::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-315::DEBUG::2012-12-09 12:11:46,901::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-315::DEBUG::2012-12-09 12:11:46,901::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-315::INFO::2012-12-09 12:11:46,902::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-315::DEBUG::2012-12-09 12:11:46,902::task::1172::TaskManager.Task::(prepare) Task=`649d2218-a14e-406a-994a-624274aeb06e`::finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-315::DEBUG::2012-12-09 12:11:46,902::task::588::TaskManager.Task::(_updateState) Task=`649d2218-a14e-406a-994a-624274aeb06e`::moving from state preparing -> state finished
>Thread-315::DEBUG::2012-12-09 12:11:46,902::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-315::DEBUG::2012-12-09 12:11:46,902::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-315::DEBUG::2012-12-09 12:11:46,902::task::978::TaskManager.Task::(_decref) Task=`649d2218-a14e-406a-994a-624274aeb06e`::ref 0 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,988::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::WARNING::2012-12-09 12:11:46,990::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:46,990::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::ERROR::2012-12-09 12:11:46,990::sp::284::Storage.StoragePool::(startSpm) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 282, in startSpm
>    self.checkBackupDomain(__securityOverride=True)
>  File "/usr/share/vdsm/storage/securable.py", line 63, in wrapper
>    return f(self, *args, **kwargs)
>  File "/usr/share/vdsm/storage/sp.py", line 1587, in checkBackupDomain
>    dom = sdCache.produce(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,000::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,004::storage_mailbox::503::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1']
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,038::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221514 s, 46.2 MB/s\n'; <rc> = 0
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,039::storage_mailbox::510::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool 4372535e-3f4d-11e2-8ba0-525400202852
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,039::sp::306::Storage.StoragePool::(startSpm) SPM mailbox ready for pool 4372535e-3f4d-11e2-8ba0-525400202852 on master domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Dummy-321::DEBUG::2012-12-09 12:11:47,039::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,040::sp::313::Storage.StoragePool::(startSpm) ended.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,047::task::588::TaskManager.Task::(_updateState) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::moving from state running -> state finished
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,050::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,050::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,051::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,051::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,052::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,052::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,053::task::900::TaskManager.Task::(_runJobs) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::Task.run: exit - success: result 
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,053::task::978::TaskManager.Task::(_decref) Task=`87f482a9-1b1a-4fe7-aa72-9cddf7717383`::ref 0 aborting False
>87f482a9-1b1a-4fe7-aa72-9cddf7717383::DEBUG::2012-12-09 12:11:47,054::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Dummy-321::DEBUG::2012-12-09 12:11:47,101::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0356068 s, 28.8 MB/s\n'; <rc> = 0
>Thread-323::DEBUG::2012-12-09 12:11:47,933::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-323::DEBUG::2012-12-09 12:11:47,936::task::588::TaskManager.Task::(_updateState) Task=`f89150c7-4868-4abc-85d0-f0d4f4c19efe`::moving from state init -> state preparing
>Thread-323::INFO::2012-12-09 12:11:47,936::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-323::DEBUG::2012-12-09 12:11:47,937::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-323::DEBUG::2012-12-09 12:11:47,937::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}
>Thread-323::INFO::2012-12-09 12:11:47,938::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-323::DEBUG::2012-12-09 12:11:47,938::task::1172::TaskManager.Task::(prepare) Task=`f89150c7-4868-4abc-85d0-f0d4f4c19efe`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '87f482a9-1b1a-4fe7-aa72-9cddf7717383'}}
>Thread-323::DEBUG::2012-12-09 12:11:47,938::task::588::TaskManager.Task::(_updateState) Task=`f89150c7-4868-4abc-85d0-f0d4f4c19efe`::moving from state preparing -> state finished
>Thread-323::DEBUG::2012-12-09 12:11:47,939::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-323::DEBUG::2012-12-09 12:11:47,939::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-323::DEBUG::2012-12-09 12:11:47,939::task::978::TaskManager.Task::(_decref) Task=`f89150c7-4868-4abc-85d0-f0d4f4c19efe`::ref 0 aborting False
>Thread-324::DEBUG::2012-12-09 12:11:47,961::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-324::DEBUG::2012-12-09 12:11:47,962::task::588::TaskManager.Task::(_updateState) Task=`a5ea1cf9-fd44-4f81-8077-d33eb958aa25`::moving from state init -> state preparing
>Thread-324::INFO::2012-12-09 12:11:47,962::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-324::INFO::2012-12-09 12:11:47,963::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-324::DEBUG::2012-12-09 12:11:47,963::task::1172::TaskManager.Task::(prepare) Task=`a5ea1cf9-fd44-4f81-8077-d33eb958aa25`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-324::DEBUG::2012-12-09 12:11:47,964::task::588::TaskManager.Task::(_updateState) Task=`a5ea1cf9-fd44-4f81-8077-d33eb958aa25`::moving from state preparing -> state finished
>Thread-324::DEBUG::2012-12-09 12:11:47,964::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-324::DEBUG::2012-12-09 12:11:47,964::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-324::DEBUG::2012-12-09 12:11:47,965::task::978::TaskManager.Task::(_decref) Task=`a5ea1cf9-fd44-4f81-8077-d33eb958aa25`::ref 0 aborting False
>Thread-325::DEBUG::2012-12-09 12:11:47,992::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-325::DEBUG::2012-12-09 12:11:47,993::task::588::TaskManager.Task::(_updateState) Task=`caaf4a01-3352-4371-8ac2-67ad8e2b55da`::moving from state init -> state preparing
>Thread-325::INFO::2012-12-09 12:11:47,994::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='87f482a9-1b1a-4fe7-aa72-9cddf7717383', spUUID=None, options=None)
>Thread-325::DEBUG::2012-12-09 12:11:47,994::taskManager::161::TaskManager::(clearTask) Entry. taskID: 87f482a9-1b1a-4fe7-aa72-9cddf7717383
>Thread-325::DEBUG::2012-12-09 12:11:47,994::taskManager::166::TaskManager::(clearTask) Return.
>Thread-325::INFO::2012-12-09 12:11:47,995::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None
>Thread-325::DEBUG::2012-12-09 12:11:47,995::task::1172::TaskManager.Task::(prepare) Task=`caaf4a01-3352-4371-8ac2-67ad8e2b55da`::finished: None
>Thread-325::DEBUG::2012-12-09 12:11:47,996::task::588::TaskManager.Task::(_updateState) Task=`caaf4a01-3352-4371-8ac2-67ad8e2b55da`::moving from state preparing -> state finished
>Thread-325::DEBUG::2012-12-09 12:11:47,996::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-325::DEBUG::2012-12-09 12:11:47,996::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-325::DEBUG::2012-12-09 12:11:47,997::task::978::TaskManager.Task::(_decref) Task=`caaf4a01-3352-4371-8ac2-67ad8e2b55da`::ref 0 aborting False
>Thread-326::DEBUG::2012-12-09 12:11:48,095::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-326::DEBUG::2012-12-09 12:11:48,096::task::588::TaskManager.Task::(_updateState) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::moving from state init -> state preparing
>Thread-326::INFO::2012-12-09 12:11:48,096::logUtils::37::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-326::DEBUG::2012-12-09 12:11:48,096::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbc6d890-8212-40e6-9b63-24f50a8fda52`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-326::DEBUG::2012-12-09 12:11:48,097::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-326::DEBUG::2012-12-09 12:11:48,097::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-326::DEBUG::2012-12-09 12:11:48,097::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbc6d890-8212-40e6-9b63-24f50a8fda52`::Granted request
>Thread-326::DEBUG::2012-12-09 12:11:48,097::task::817::TaskManager.Task::(resourceAcquired) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (exclusive)
>Thread-326::DEBUG::2012-12-09 12:11:48,098::task::978::TaskManager.Task::(_decref) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::ref 1 aborting False
>Thread-326::DEBUG::2012-12-09 12:11:48,098::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`e400a243-ac3e-447b-b13a-4a04a4fd5a21`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-326::DEBUG::2012-12-09 12:11:48,098::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' for lock type 'exclusive'
>Thread-326::DEBUG::2012-12-09 12:11:48,098::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free. Now locking as 'exclusive' (1 active user)
>Thread-326::DEBUG::2012-12-09 12:11:48,099::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`e400a243-ac3e-447b-b13a-4a04a4fd5a21`::Granted request
>Thread-326::DEBUG::2012-12-09 12:11:48,099::task::817::TaskManager.Task::(resourceAcquired) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::_resourcesAcquired: Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (exclusive)
>Thread-326::DEBUG::2012-12-09 12:11:48,099::task::978::TaskManager.Task::(_decref) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::ref 1 aborting False
>Thread-326::INFO::2012-12-09 12:11:48,099::sp::1049::Storage.StoragePool::(activateSD) sdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 spUUID=4372535e-3f4d-11e2-8ba0-525400202852
>Thread-326::INFO::2012-12-09 12:11:48,100::logUtils::39::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None
>Thread-326::DEBUG::2012-12-09 12:11:48,100::task::1172::TaskManager.Task::(prepare) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::finished: None
>Thread-326::DEBUG::2012-12-09 12:11:48,100::task::588::TaskManager.Task::(_updateState) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::moving from state preparing -> state finished
>Thread-326::DEBUG::2012-12-09 12:11:48,100::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': < ResourceRef 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', isValid: 'True' obj: 'None'>, 'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-326::DEBUG::2012-12-09 12:11:48,100::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-326::DEBUG::2012-12-09 12:11:48,100::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'
>Thread-326::DEBUG::2012-12-09 12:11:48,101::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (0 active users)
>Thread-326::DEBUG::2012-12-09 12:11:48,101::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free, finding out if anyone is waiting for it.
>Thread-326::DEBUG::2012-12-09 12:11:48,101::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', Clearing records.
>Thread-326::DEBUG::2012-12-09 12:11:48,101::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-326::DEBUG::2012-12-09 12:11:48,102::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-326::DEBUG::2012-12-09 12:11:48,102::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-326::DEBUG::2012-12-09 12:11:48,102::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-326::DEBUG::2012-12-09 12:11:48,102::task::978::TaskManager.Task::(_decref) Task=`e1e54288-87a4-4581-97c8-81472f7f0753`::ref 0 aborting False
>Thread-327::DEBUG::2012-12-09 12:11:48,121::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-327::DEBUG::2012-12-09 12:11:48,121::task::588::TaskManager.Task::(_updateState) Task=`fdceeaa7-8061-4d53-94d5-467cb3de9950`::moving from state init -> state preparing
>Thread-327::INFO::2012-12-09 12:11:48,122::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-327::INFO::2012-12-09 12:11:48,122::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-327::DEBUG::2012-12-09 12:11:48,122::task::1172::TaskManager.Task::(prepare) Task=`fdceeaa7-8061-4d53-94d5-467cb3de9950`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-327::DEBUG::2012-12-09 12:11:48,122::task::588::TaskManager.Task::(_updateState) Task=`fdceeaa7-8061-4d53-94d5-467cb3de9950`::moving from state preparing -> state finished
>Thread-327::DEBUG::2012-12-09 12:11:48,122::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-327::DEBUG::2012-12-09 12:11:48,123::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-327::DEBUG::2012-12-09 12:11:48,123::task::978::TaskManager.Task::(_decref) Task=`fdceeaa7-8061-4d53-94d5-467cb3de9950`::ref 0 aborting False
>Thread-328::DEBUG::2012-12-09 12:11:48,138::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-328::DEBUG::2012-12-09 12:11:48,139::task::588::TaskManager.Task::(_updateState) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::moving from state init -> state preparing
>Thread-328::INFO::2012-12-09 12:11:48,139::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-328::DEBUG::2012-12-09 12:11:48,140::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79811666-2c19-41ec-b144-0d91d0d008d6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-328::DEBUG::2012-12-09 12:11:48,140::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-328::DEBUG::2012-12-09 12:11:48,140::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-328::DEBUG::2012-12-09 12:11:48,140::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79811666-2c19-41ec-b144-0d91d0d008d6`::Granted request
>Thread-328::DEBUG::2012-12-09 12:11:48,141::task::817::TaskManager.Task::(resourceAcquired) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-328::DEBUG::2012-12-09 12:11:48,141::task::978::TaskManager.Task::(_decref) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::ref 1 aborting False
>Thread-328::DEBUG::2012-12-09 12:11:48,142::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-328::DEBUG::2012-12-09 12:11:48,142::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-329::DEBUG::2012-12-09 12:11:48,168::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-329::DEBUG::2012-12-09 12:11:48,168::task::588::TaskManager.Task::(_updateState) Task=`4fd404ee-4687-4c02-a8c2-e831cf37fd62`::moving from state init -> state preparing
>Thread-329::INFO::2012-12-09 12:11:48,169::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', hostID=2, scsiKey='4372535e-3f4d-11e2-8ba0-525400202852', msdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', masterVersion=1, options=None)
>Thread-329::DEBUG::2012-12-09 12:11:48,170::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4e3629b2-3b3f-4739-8c7f-049d37fd5cbd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-329::DEBUG::2012-12-09 12:11:48,170::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-329::DEBUG::2012-12-09 12:11:48,170::resourceManager::504::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-329::DEBUG::2012-12-09 12:11:48,171::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4e3629b2-3b3f-4739-8c7f-049d37fd5cbd`::Granted request
>Thread-329::DEBUG::2012-12-09 12:11:48,171::sp::1539::Storage.StoragePool::(getMasterDomain) Master domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 verified, version 1
>Thread-329::DEBUG::2012-12-09 12:11:48,172::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-329::DEBUG::2012-12-09 12:11:48,172::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (1 active users)
>Thread-329::INFO::2012-12-09 12:11:48,172::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: None
>Thread-329::DEBUG::2012-12-09 12:11:48,173::task::1172::TaskManager.Task::(prepare) Task=`4fd404ee-4687-4c02-a8c2-e831cf37fd62`::finished: None
>Thread-329::DEBUG::2012-12-09 12:11:48,173::task::588::TaskManager.Task::(_updateState) Task=`4fd404ee-4687-4c02-a8c2-e831cf37fd62`::moving from state preparing -> state finished
>Thread-329::DEBUG::2012-12-09 12:11:48,173::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-329::DEBUG::2012-12-09 12:11:48,174::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-329::DEBUG::2012-12-09 12:11:48,174::task::978::TaskManager.Task::(_decref) Task=`4fd404ee-4687-4c02-a8c2-e831cf37fd62`::ref 0 aborting False
>Thread-328::DEBUG::2012-12-09 12:11:48,247::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-328::WARNING::2012-12-09 12:11:48,248::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-328::DEBUG::2012-12-09 12:11:48,248::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-328::WARNING::2012-12-09 12:11:48,249::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-328::INFO::2012-12-09 12:11:48,249::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-328::DEBUG::2012-12-09 12:11:48,249::task::1172::TaskManager.Task::(prepare) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-328::DEBUG::2012-12-09 12:11:48,250::task::588::TaskManager.Task::(_updateState) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::moving from state preparing -> state finished
>Thread-328::DEBUG::2012-12-09 12:11:48,250::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-328::DEBUG::2012-12-09 12:11:48,250::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-328::DEBUG::2012-12-09 12:11:48,250::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-328::DEBUG::2012-12-09 12:11:48,250::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-328::DEBUG::2012-12-09 12:11:48,251::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-328::DEBUG::2012-12-09 12:11:48,251::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-328::DEBUG::2012-12-09 12:11:48,251::task::978::TaskManager.Task::(_decref) Task=`7d2c311f-8b2e-49a3-b200-60b1ee278f89`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:11:49,107::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:49,161::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237039 s, 43.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:11:51,168::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:51,214::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227306 s, 45.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:11:53,221::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:53,261::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168873 s, 60.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:11:54,904::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:11:54,942::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:11:54,943::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:11:55,012::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:11:55,012::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:11:55,033::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000491911 s, 8.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:11:55,044::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:11:55,045::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:11:55,045::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:11:55,046::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:11:55,267::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:55,305::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158866 s, 64.5 MB/s\n'; <rc> = 0
>Thread-333::DEBUG::2012-12-09 12:11:55,895::task::588::TaskManager.Task::(_updateState) Task=`4873738b-cfaa-4349-9145-3e47c80bab0f`::moving from state init -> state preparing
>Thread-333::INFO::2012-12-09 12:11:55,896::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-333::INFO::2012-12-09 12:11:55,897::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0215618610382', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-333::DEBUG::2012-12-09 12:11:55,897::task::1172::TaskManager.Task::(prepare) Task=`4873738b-cfaa-4349-9145-3e47c80bab0f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0215618610382', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-333::DEBUG::2012-12-09 12:11:55,898::task::588::TaskManager.Task::(_updateState) Task=`4873738b-cfaa-4349-9145-3e47c80bab0f`::moving from state preparing -> state finished
>Thread-333::DEBUG::2012-12-09 12:11:55,898::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-333::DEBUG::2012-12-09 12:11:55,898::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-333::DEBUG::2012-12-09 12:11:55,899::task::978::TaskManager.Task::(_decref) Task=`4873738b-cfaa-4349-9145-3e47c80bab0f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:11:57,312::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:57,350::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169216 s, 60.5 MB/s\n'; <rc> = 0
>Thread-336::DEBUG::2012-12-09 12:11:58,728::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-336::DEBUG::2012-12-09 12:11:58,728::task::588::TaskManager.Task::(_updateState) Task=`76d54401-8bf2-4597-a010-c2b3f6194ff8`::moving from state init -> state preparing
>Thread-336::INFO::2012-12-09 12:11:58,728::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-336::INFO::2012-12-09 12:11:58,729::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-336::DEBUG::2012-12-09 12:11:58,729::task::1172::TaskManager.Task::(prepare) Task=`76d54401-8bf2-4597-a010-c2b3f6194ff8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-336::DEBUG::2012-12-09 12:11:58,729::task::588::TaskManager.Task::(_updateState) Task=`76d54401-8bf2-4597-a010-c2b3f6194ff8`::moving from state preparing -> state finished
>Thread-336::DEBUG::2012-12-09 12:11:58,729::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-336::DEBUG::2012-12-09 12:11:58,729::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-336::DEBUG::2012-12-09 12:11:58,730::task::978::TaskManager.Task::(_decref) Task=`76d54401-8bf2-4597-a010-c2b3f6194ff8`::ref 0 aborting False
>Thread-337::DEBUG::2012-12-09 12:11:58,752::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-337::DEBUG::2012-12-09 12:11:58,753::task::588::TaskManager.Task::(_updateState) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::moving from state init -> state preparing
>Thread-337::INFO::2012-12-09 12:11:58,753::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-337::DEBUG::2012-12-09 12:11:58,754::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0aa9eb9f-d0ad-48c3-954f-61ef8accd137`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-337::DEBUG::2012-12-09 12:11:58,755::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-337::DEBUG::2012-12-09 12:11:58,755::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-337::DEBUG::2012-12-09 12:11:58,756::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0aa9eb9f-d0ad-48c3-954f-61ef8accd137`::Granted request
>Thread-337::DEBUG::2012-12-09 12:11:58,756::task::817::TaskManager.Task::(resourceAcquired) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-337::DEBUG::2012-12-09 12:11:58,757::task::978::TaskManager.Task::(_decref) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::ref 1 aborting False
>Thread-337::DEBUG::2012-12-09 12:11:58,758::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-337::DEBUG::2012-12-09 12:11:58,759::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-337::DEBUG::2012-12-09 12:11:58,873::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-337::WARNING::2012-12-09 12:11:58,875::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-337::DEBUG::2012-12-09 12:11:58,876::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-337::WARNING::2012-12-09 12:11:58,877::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-337::INFO::2012-12-09 12:11:58,878::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-337::DEBUG::2012-12-09 12:11:58,878::task::1172::TaskManager.Task::(prepare) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-337::DEBUG::2012-12-09 12:11:58,879::task::588::TaskManager.Task::(_updateState) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::moving from state preparing -> state finished
>Thread-337::DEBUG::2012-12-09 12:11:58,879::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-337::DEBUG::2012-12-09 12:11:58,880::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-337::DEBUG::2012-12-09 12:11:58,880::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-337::DEBUG::2012-12-09 12:11:58,880::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-337::DEBUG::2012-12-09 12:11:58,880::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-337::DEBUG::2012-12-09 12:11:58,880::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-337::DEBUG::2012-12-09 12:11:58,881::task::978::TaskManager.Task::(_decref) Task=`222c9502-3697-4d99-9ce3-01f187c7a0f6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:11:59,357::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:11:59,396::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163312 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:01,403::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:01,453::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269578 s, 38.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:03,460::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:03,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224072 s, 45.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:05,047::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:05,048::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:05,080::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:05,103::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000656584 s, 6.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:05,161::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:05,161::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:05,162::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:05,162::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:05,514::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:05,565::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258089 s, 39.7 MB/s\n'; <rc> = 0
>Thread-341::DEBUG::2012-12-09 12:12:06,254::task::588::TaskManager.Task::(_updateState) Task=`0cbea3f9-f546-436c-9925-5e4502dec994`::moving from state init -> state preparing
>Thread-341::INFO::2012-12-09 12:12:06,254::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-341::INFO::2012-12-09 12:12:06,255::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.023313999176', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-341::DEBUG::2012-12-09 12:12:06,255::task::1172::TaskManager.Task::(prepare) Task=`0cbea3f9-f546-436c-9925-5e4502dec994`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.023313999176', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-341::DEBUG::2012-12-09 12:12:06,256::task::588::TaskManager.Task::(_updateState) Task=`0cbea3f9-f546-436c-9925-5e4502dec994`::moving from state preparing -> state finished
>Thread-341::DEBUG::2012-12-09 12:12:06,256::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-341::DEBUG::2012-12-09 12:12:06,257::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-341::DEBUG::2012-12-09 12:12:06,257::task::978::TaskManager.Task::(_decref) Task=`0cbea3f9-f546-436c-9925-5e4502dec994`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:07,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:07,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168419 s, 60.8 MB/s\n'; <rc> = 0
>Thread-344::DEBUG::2012-12-09 12:12:08,995::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-344::DEBUG::2012-12-09 12:12:08,996::task::588::TaskManager.Task::(_updateState) Task=`42593c74-5fd3-4c9d-ac68-653f415f1556`::moving from state init -> state preparing
>Thread-344::INFO::2012-12-09 12:12:08,996::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-344::INFO::2012-12-09 12:12:08,996::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-344::DEBUG::2012-12-09 12:12:08,997::task::1172::TaskManager.Task::(prepare) Task=`42593c74-5fd3-4c9d-ac68-653f415f1556`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-344::DEBUG::2012-12-09 12:12:08,997::task::588::TaskManager.Task::(_updateState) Task=`42593c74-5fd3-4c9d-ac68-653f415f1556`::moving from state preparing -> state finished
>Thread-344::DEBUG::2012-12-09 12:12:08,997::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-344::DEBUG::2012-12-09 12:12:08,997::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-344::DEBUG::2012-12-09 12:12:08,997::task::978::TaskManager.Task::(_decref) Task=`42593c74-5fd3-4c9d-ac68-653f415f1556`::ref 0 aborting False
>Thread-345::DEBUG::2012-12-09 12:12:09,013::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-345::DEBUG::2012-12-09 12:12:09,013::task::588::TaskManager.Task::(_updateState) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::moving from state init -> state preparing
>Thread-345::INFO::2012-12-09 12:12:09,013::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-345::DEBUG::2012-12-09 12:12:09,014::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4689c70-0ba2-4898-84d1-da1b51422889`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-345::DEBUG::2012-12-09 12:12:09,014::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-345::DEBUG::2012-12-09 12:12:09,014::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-345::DEBUG::2012-12-09 12:12:09,014::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d4689c70-0ba2-4898-84d1-da1b51422889`::Granted request
>Thread-345::DEBUG::2012-12-09 12:12:09,015::task::817::TaskManager.Task::(resourceAcquired) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-345::DEBUG::2012-12-09 12:12:09,015::task::978::TaskManager.Task::(_decref) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::ref 1 aborting False
>Thread-345::DEBUG::2012-12-09 12:12:09,015::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-345::DEBUG::2012-12-09 12:12:09,016::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-345::DEBUG::2012-12-09 12:12:09,124::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-345::WARNING::2012-12-09 12:12:09,126::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-345::DEBUG::2012-12-09 12:12:09,127::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-345::WARNING::2012-12-09 12:12:09,128::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-345::INFO::2012-12-09 12:12:09,128::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-345::DEBUG::2012-12-09 12:12:09,129::task::1172::TaskManager.Task::(prepare) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-345::DEBUG::2012-12-09 12:12:09,129::task::588::TaskManager.Task::(_updateState) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::moving from state preparing -> state finished
>Thread-345::DEBUG::2012-12-09 12:12:09,130::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-345::DEBUG::2012-12-09 12:12:09,130::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-345::DEBUG::2012-12-09 12:12:09,130::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-345::DEBUG::2012-12-09 12:12:09,130::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-345::DEBUG::2012-12-09 12:12:09,130::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-345::DEBUG::2012-12-09 12:12:09,131::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-345::DEBUG::2012-12-09 12:12:09,131::task::978::TaskManager.Task::(_decref) Task=`995bb487-fe15-453d-8892-af8f2d20722b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:09,617::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:09,659::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172803 s, 59.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:11,666::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:11,704::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162764 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:13,711::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:13,750::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171193 s, 59.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:12:15,107::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:15,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000502832 s, 8.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:15,163::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:15,164::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:12:15,262::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:15,263::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:15,263::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:15,264::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:15,756::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:15,805::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227451 s, 45.0 MB/s\n'; <rc> = 0
>Thread-349::DEBUG::2012-12-09 12:12:16,542::task::588::TaskManager.Task::(_updateState) Task=`e961d208-a218-46a8-9e03-7ef7cd01fbb5`::moving from state init -> state preparing
>Thread-349::INFO::2012-12-09 12:12:16,543::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-349::INFO::2012-12-09 12:12:16,544::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113868713379', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-349::DEBUG::2012-12-09 12:12:16,544::task::1172::TaskManager.Task::(prepare) Task=`e961d208-a218-46a8-9e03-7ef7cd01fbb5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113868713379', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-349::DEBUG::2012-12-09 12:12:16,544::task::588::TaskManager.Task::(_updateState) Task=`e961d208-a218-46a8-9e03-7ef7cd01fbb5`::moving from state preparing -> state finished
>Thread-349::DEBUG::2012-12-09 12:12:16,545::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-349::DEBUG::2012-12-09 12:12:16,545::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-349::DEBUG::2012-12-09 12:12:16,546::task::978::TaskManager.Task::(_decref) Task=`e961d208-a218-46a8-9e03-7ef7cd01fbb5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:17,811::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:17,850::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167389 s, 61.2 MB/s\n'; <rc> = 0
>Thread-352::DEBUG::2012-12-09 12:12:19,443::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-352::DEBUG::2012-12-09 12:12:19,444::task::588::TaskManager.Task::(_updateState) Task=`500a12d7-4ac5-4dc7-8085-395550e8811d`::moving from state init -> state preparing
>Thread-352::INFO::2012-12-09 12:12:19,444::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-352::INFO::2012-12-09 12:12:19,444::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-352::DEBUG::2012-12-09 12:12:19,444::task::1172::TaskManager.Task::(prepare) Task=`500a12d7-4ac5-4dc7-8085-395550e8811d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-352::DEBUG::2012-12-09 12:12:19,444::task::588::TaskManager.Task::(_updateState) Task=`500a12d7-4ac5-4dc7-8085-395550e8811d`::moving from state preparing -> state finished
>Thread-352::DEBUG::2012-12-09 12:12:19,445::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-352::DEBUG::2012-12-09 12:12:19,445::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-352::DEBUG::2012-12-09 12:12:19,445::task::978::TaskManager.Task::(_decref) Task=`500a12d7-4ac5-4dc7-8085-395550e8811d`::ref 0 aborting False
>Thread-353::DEBUG::2012-12-09 12:12:19,459::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-353::DEBUG::2012-12-09 12:12:19,459::task::588::TaskManager.Task::(_updateState) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::moving from state init -> state preparing
>Thread-353::INFO::2012-12-09 12:12:19,460::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-353::DEBUG::2012-12-09 12:12:19,460::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`88b1fa3e-1213-4556-94ca-a9cb8e19a5a8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-353::DEBUG::2012-12-09 12:12:19,460::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-353::DEBUG::2012-12-09 12:12:19,460::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-353::DEBUG::2012-12-09 12:12:19,460::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`88b1fa3e-1213-4556-94ca-a9cb8e19a5a8`::Granted request
>Thread-353::DEBUG::2012-12-09 12:12:19,461::task::817::TaskManager.Task::(resourceAcquired) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-353::DEBUG::2012-12-09 12:12:19,461::task::978::TaskManager.Task::(_decref) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::ref 1 aborting False
>Thread-353::DEBUG::2012-12-09 12:12:19,462::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-353::DEBUG::2012-12-09 12:12:19,462::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-353::DEBUG::2012-12-09 12:12:19,560::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-353::WARNING::2012-12-09 12:12:19,561::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-353::DEBUG::2012-12-09 12:12:19,562::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-353::WARNING::2012-12-09 12:12:19,562::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-353::INFO::2012-12-09 12:12:19,562::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-353::DEBUG::2012-12-09 12:12:19,563::task::1172::TaskManager.Task::(prepare) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-353::DEBUG::2012-12-09 12:12:19,563::task::588::TaskManager.Task::(_updateState) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::moving from state preparing -> state finished
>Thread-353::DEBUG::2012-12-09 12:12:19,563::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-353::DEBUG::2012-12-09 12:12:19,563::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-353::DEBUG::2012-12-09 12:12:19,563::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-353::DEBUG::2012-12-09 12:12:19,564::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-353::DEBUG::2012-12-09 12:12:19,564::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-353::DEBUG::2012-12-09 12:12:19,564::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-353::DEBUG::2012-12-09 12:12:19,564::task::978::TaskManager.Task::(_decref) Task=`9c463e28-c720-46f8-8f20-315506f6f210`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:19,857::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:19,896::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161524 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:21,903::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:21,942::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164675 s, 62.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:23,949::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:23,988::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166494 s, 61.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:12:25,121::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:25,132::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000523409 s, 7.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:25,265::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:25,265::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:12:25,366::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:25,367::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:25,368::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:25,368::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:25,994::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:26,035::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168894 s, 60.6 MB/s\n'; <rc> = 0
>Thread-357::DEBUG::2012-12-09 12:12:26,770::task::588::TaskManager.Task::(_updateState) Task=`6e81b1d5-7206-49d6-be36-623ae05d6cd1`::moving from state init -> state preparing
>Thread-357::INFO::2012-12-09 12:12:26,771::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-357::INFO::2012-12-09 12:12:26,771::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116000175476', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-357::DEBUG::2012-12-09 12:12:26,771::task::1172::TaskManager.Task::(prepare) Task=`6e81b1d5-7206-49d6-be36-623ae05d6cd1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116000175476', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-357::DEBUG::2012-12-09 12:12:26,772::task::588::TaskManager.Task::(_updateState) Task=`6e81b1d5-7206-49d6-be36-623ae05d6cd1`::moving from state preparing -> state finished
>Thread-357::DEBUG::2012-12-09 12:12:26,772::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-357::DEBUG::2012-12-09 12:12:26,772::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-357::DEBUG::2012-12-09 12:12:26,772::task::978::TaskManager.Task::(_decref) Task=`6e81b1d5-7206-49d6-be36-623ae05d6cd1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:28,041::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:28,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021884 s, 46.8 MB/s\n'; <rc> = 0
>Thread-360::DEBUG::2012-12-09 12:12:29,833::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-360::DEBUG::2012-12-09 12:12:29,834::task::588::TaskManager.Task::(_updateState) Task=`63777e76-5326-4625-800c-d395bb583a92`::moving from state init -> state preparing
>Thread-360::INFO::2012-12-09 12:12:29,834::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-360::INFO::2012-12-09 12:12:29,835::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-360::DEBUG::2012-12-09 12:12:29,835::task::1172::TaskManager.Task::(prepare) Task=`63777e76-5326-4625-800c-d395bb583a92`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-360::DEBUG::2012-12-09 12:12:29,835::task::588::TaskManager.Task::(_updateState) Task=`63777e76-5326-4625-800c-d395bb583a92`::moving from state preparing -> state finished
>Thread-360::DEBUG::2012-12-09 12:12:29,835::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-360::DEBUG::2012-12-09 12:12:29,835::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-360::DEBUG::2012-12-09 12:12:29,836::task::978::TaskManager.Task::(_decref) Task=`63777e76-5326-4625-800c-d395bb583a92`::ref 0 aborting False
>Thread-361::DEBUG::2012-12-09 12:12:29,848::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-361::DEBUG::2012-12-09 12:12:29,849::task::588::TaskManager.Task::(_updateState) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::moving from state init -> state preparing
>Thread-361::INFO::2012-12-09 12:12:29,849::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-361::DEBUG::2012-12-09 12:12:29,849::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6c327d7-2a1f-40fd-8a5b-42a4beb1bc04`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-361::DEBUG::2012-12-09 12:12:29,850::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-361::DEBUG::2012-12-09 12:12:29,850::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-361::DEBUG::2012-12-09 12:12:29,850::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6c327d7-2a1f-40fd-8a5b-42a4beb1bc04`::Granted request
>Thread-361::DEBUG::2012-12-09 12:12:29,850::task::817::TaskManager.Task::(resourceAcquired) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-361::DEBUG::2012-12-09 12:12:29,851::task::978::TaskManager.Task::(_decref) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::ref 1 aborting False
>Thread-361::DEBUG::2012-12-09 12:12:29,851::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-361::DEBUG::2012-12-09 12:12:29,852::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-361::DEBUG::2012-12-09 12:12:29,949::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-361::WARNING::2012-12-09 12:12:29,950::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-361::DEBUG::2012-12-09 12:12:29,950::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-361::WARNING::2012-12-09 12:12:29,951::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-361::INFO::2012-12-09 12:12:29,951::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-361::DEBUG::2012-12-09 12:12:29,951::task::1172::TaskManager.Task::(prepare) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-361::DEBUG::2012-12-09 12:12:29,952::task::588::TaskManager.Task::(_updateState) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::moving from state preparing -> state finished
>Thread-361::DEBUG::2012-12-09 12:12:29,952::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-361::DEBUG::2012-12-09 12:12:29,952::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-361::DEBUG::2012-12-09 12:12:29,952::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-361::DEBUG::2012-12-09 12:12:29,952::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-361::DEBUG::2012-12-09 12:12:29,953::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-361::DEBUG::2012-12-09 12:12:29,953::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-361::DEBUG::2012-12-09 12:12:29,953::task::978::TaskManager.Task::(_decref) Task=`05db0b7c-20d3-4df9-86d4-02b7b1a312fb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:30,094::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:30,134::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163257 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:32,140::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:32,180::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166948 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:34,187::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:34,240::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176126 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:12:35,135::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:35,147::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00067814 s, 6.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:35,369::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:35,370::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:12:35,467::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:35,468::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:35,468::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:35,469::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:36,247::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:36,296::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234438 s, 43.7 MB/s\n'; <rc> = 0
>Thread-365::DEBUG::2012-12-09 12:12:37,059::task::588::TaskManager.Task::(_updateState) Task=`95cc9c57-4cde-4ad8-a945-95f49cd22b94`::moving from state init -> state preparing
>Thread-365::INFO::2012-12-09 12:12:37,059::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-365::INFO::2012-12-09 12:12:37,060::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121278762817', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-365::DEBUG::2012-12-09 12:12:37,060::task::1172::TaskManager.Task::(prepare) Task=`95cc9c57-4cde-4ad8-a945-95f49cd22b94`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121278762817', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-365::DEBUG::2012-12-09 12:12:37,061::task::588::TaskManager.Task::(_updateState) Task=`95cc9c57-4cde-4ad8-a945-95f49cd22b94`::moving from state preparing -> state finished
>Thread-365::DEBUG::2012-12-09 12:12:37,061::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-365::DEBUG::2012-12-09 12:12:37,062::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-365::DEBUG::2012-12-09 12:12:37,062::task::978::TaskManager.Task::(_decref) Task=`95cc9c57-4cde-4ad8-a945-95f49cd22b94`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:38,302::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:38,354::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026525 s, 38.6 MB/s\n'; <rc> = 0
>Thread-368::DEBUG::2012-12-09 12:12:40,064::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-368::DEBUG::2012-12-09 12:12:40,065::task::588::TaskManager.Task::(_updateState) Task=`501e01b5-8671-4884-850d-5aab212acabf`::moving from state init -> state preparing
>Thread-368::INFO::2012-12-09 12:12:40,066::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-368::INFO::2012-12-09 12:12:40,066::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-368::DEBUG::2012-12-09 12:12:40,067::task::1172::TaskManager.Task::(prepare) Task=`501e01b5-8671-4884-850d-5aab212acabf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-368::DEBUG::2012-12-09 12:12:40,067::task::588::TaskManager.Task::(_updateState) Task=`501e01b5-8671-4884-850d-5aab212acabf`::moving from state preparing -> state finished
>Thread-368::DEBUG::2012-12-09 12:12:40,068::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-368::DEBUG::2012-12-09 12:12:40,068::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-368::DEBUG::2012-12-09 12:12:40,068::task::978::TaskManager.Task::(_decref) Task=`501e01b5-8671-4884-850d-5aab212acabf`::ref 0 aborting False
>Thread-369::DEBUG::2012-12-09 12:12:40,092::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-369::DEBUG::2012-12-09 12:12:40,093::task::588::TaskManager.Task::(_updateState) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::moving from state init -> state preparing
>Thread-369::INFO::2012-12-09 12:12:40,093::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-369::DEBUG::2012-12-09 12:12:40,093::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ec06d48b-f3a1-4858-9daf-38a9db6a135f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-369::DEBUG::2012-12-09 12:12:40,094::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-369::DEBUG::2012-12-09 12:12:40,094::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-369::DEBUG::2012-12-09 12:12:40,094::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ec06d48b-f3a1-4858-9daf-38a9db6a135f`::Granted request
>Thread-369::DEBUG::2012-12-09 12:12:40,094::task::817::TaskManager.Task::(resourceAcquired) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-369::DEBUG::2012-12-09 12:12:40,095::task::978::TaskManager.Task::(_decref) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::ref 1 aborting False
>Thread-369::DEBUG::2012-12-09 12:12:40,095::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-369::DEBUG::2012-12-09 12:12:40,096::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-369::DEBUG::2012-12-09 12:12:40,196::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-369::WARNING::2012-12-09 12:12:40,197::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-369::DEBUG::2012-12-09 12:12:40,197::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-369::WARNING::2012-12-09 12:12:40,198::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-369::INFO::2012-12-09 12:12:40,198::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-369::DEBUG::2012-12-09 12:12:40,198::task::1172::TaskManager.Task::(prepare) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-369::DEBUG::2012-12-09 12:12:40,199::task::588::TaskManager.Task::(_updateState) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::moving from state preparing -> state finished
>Thread-369::DEBUG::2012-12-09 12:12:40,199::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-369::DEBUG::2012-12-09 12:12:40,199::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-369::DEBUG::2012-12-09 12:12:40,199::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-369::DEBUG::2012-12-09 12:12:40,200::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-369::DEBUG::2012-12-09 12:12:40,200::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-369::DEBUG::2012-12-09 12:12:40,200::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-369::DEBUG::2012-12-09 12:12:40,200::task::978::TaskManager.Task::(_decref) Task=`1453db6f-918f-4ddc-81f2-58f7e8d2cb40`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:40,361::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:40,413::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276303 s, 37.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:42,420::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:42,466::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225309 s, 45.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:44,473::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:44,524::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269224 s, 38.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:12:45,150::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:45,162::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000629016 s, 6.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:45,469::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:45,470::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:12:45,570::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:45,571::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:45,571::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:45,572::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:46,531::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:46,583::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272959 s, 37.5 MB/s\n'; <rc> = 0
>Thread-373::DEBUG::2012-12-09 12:12:47,454::task::588::TaskManager.Task::(_updateState) Task=`27e979d0-d367-4fb3-93c2-58e739c04fc0`::moving from state init -> state preparing
>Thread-373::INFO::2012-12-09 12:12:47,454::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-373::INFO::2012-12-09 12:12:47,454::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133628845215', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-373::DEBUG::2012-12-09 12:12:47,455::task::1172::TaskManager.Task::(prepare) Task=`27e979d0-d367-4fb3-93c2-58e739c04fc0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133628845215', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-373::DEBUG::2012-12-09 12:12:47,455::task::588::TaskManager.Task::(_updateState) Task=`27e979d0-d367-4fb3-93c2-58e739c04fc0`::moving from state preparing -> state finished
>Thread-373::DEBUG::2012-12-09 12:12:47,455::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-373::DEBUG::2012-12-09 12:12:47,455::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-373::DEBUG::2012-12-09 12:12:47,456::task::978::TaskManager.Task::(_decref) Task=`27e979d0-d367-4fb3-93c2-58e739c04fc0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:48,590::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:48,636::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216083 s, 47.4 MB/s\n'; <rc> = 0
>Thread-376::DEBUG::2012-12-09 12:12:50,324::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-376::DEBUG::2012-12-09 12:12:50,324::task::588::TaskManager.Task::(_updateState) Task=`584bae41-283b-4e4f-958a-a658681d3a72`::moving from state init -> state preparing
>Thread-376::INFO::2012-12-09 12:12:50,325::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-376::INFO::2012-12-09 12:12:50,325::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-376::DEBUG::2012-12-09 12:12:50,325::task::1172::TaskManager.Task::(prepare) Task=`584bae41-283b-4e4f-958a-a658681d3a72`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-376::DEBUG::2012-12-09 12:12:50,326::task::588::TaskManager.Task::(_updateState) Task=`584bae41-283b-4e4f-958a-a658681d3a72`::moving from state preparing -> state finished
>Thread-376::DEBUG::2012-12-09 12:12:50,326::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-376::DEBUG::2012-12-09 12:12:50,326::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-376::DEBUG::2012-12-09 12:12:50,326::task::978::TaskManager.Task::(_decref) Task=`584bae41-283b-4e4f-958a-a658681d3a72`::ref 0 aborting False
>Thread-377::DEBUG::2012-12-09 12:12:50,341::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-377::DEBUG::2012-12-09 12:12:50,342::task::588::TaskManager.Task::(_updateState) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::moving from state init -> state preparing
>Thread-377::INFO::2012-12-09 12:12:50,342::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-377::DEBUG::2012-12-09 12:12:50,343::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dd40311c-3a9c-4671-91e3-00a57e2ca594`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-377::DEBUG::2012-12-09 12:12:50,343::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-377::DEBUG::2012-12-09 12:12:50,344::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-377::DEBUG::2012-12-09 12:12:50,344::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dd40311c-3a9c-4671-91e3-00a57e2ca594`::Granted request
>Thread-377::DEBUG::2012-12-09 12:12:50,345::task::817::TaskManager.Task::(resourceAcquired) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-377::DEBUG::2012-12-09 12:12:50,345::task::978::TaskManager.Task::(_decref) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::ref 1 aborting False
>Thread-377::DEBUG::2012-12-09 12:12:50,347::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-377::DEBUG::2012-12-09 12:12:50,348::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-377::DEBUG::2012-12-09 12:12:50,458::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-377::WARNING::2012-12-09 12:12:50,460::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-377::DEBUG::2012-12-09 12:12:50,460::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-377::WARNING::2012-12-09 12:12:50,461::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-377::INFO::2012-12-09 12:12:50,461::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-377::DEBUG::2012-12-09 12:12:50,461::task::1172::TaskManager.Task::(prepare) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-377::DEBUG::2012-12-09 12:12:50,461::task::588::TaskManager.Task::(_updateState) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::moving from state preparing -> state finished
>Thread-377::DEBUG::2012-12-09 12:12:50,461::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-377::DEBUG::2012-12-09 12:12:50,462::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-377::DEBUG::2012-12-09 12:12:50,462::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-377::DEBUG::2012-12-09 12:12:50,462::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-377::DEBUG::2012-12-09 12:12:50,462::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-377::DEBUG::2012-12-09 12:12:50,462::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-377::DEBUG::2012-12-09 12:12:50,463::task::978::TaskManager.Task::(_decref) Task=`2dc6d500-adda-4a06-859d-e92e09c4634b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:50,643::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:50,688::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201425 s, 50.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:52,695::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:52,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164995 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:12:54,741::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:54,789::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217857 s, 47.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:12:55,168::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:12:55,183::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00088494 s, 4.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:12:55,573::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:12:55,573::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:12:55,672::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:12:55,673::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:12:55,673::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:12:55,673::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:12:56,796::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:56,844::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238506 s, 42.9 MB/s\n'; <rc> = 0
>Thread-381::DEBUG::2012-12-09 12:12:57,716::task::588::TaskManager.Task::(_updateState) Task=`ec6a50fd-fb3b-4d69-90bb-8740da502d48`::moving from state init -> state preparing
>Thread-381::INFO::2012-12-09 12:12:57,717::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-381::INFO::2012-12-09 12:12:57,717::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161871910095', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-381::DEBUG::2012-12-09 12:12:57,717::task::1172::TaskManager.Task::(prepare) Task=`ec6a50fd-fb3b-4d69-90bb-8740da502d48`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161871910095', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-381::DEBUG::2012-12-09 12:12:57,718::task::588::TaskManager.Task::(_updateState) Task=`ec6a50fd-fb3b-4d69-90bb-8740da502d48`::moving from state preparing -> state finished
>Thread-381::DEBUG::2012-12-09 12:12:57,718::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-381::DEBUG::2012-12-09 12:12:57,718::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-381::DEBUG::2012-12-09 12:12:57,719::task::978::TaskManager.Task::(_decref) Task=`ec6a50fd-fb3b-4d69-90bb-8740da502d48`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:12:58,851::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:12:58,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279481 s, 36.6 MB/s\n'; <rc> = 0
>Thread-384::DEBUG::2012-12-09 12:13:00,602::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-384::DEBUG::2012-12-09 12:13:00,603::task::588::TaskManager.Task::(_updateState) Task=`222dbfc4-bac5-4a37-baf3-f3e2726551aa`::moving from state init -> state preparing
>Thread-384::INFO::2012-12-09 12:13:00,603::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-384::INFO::2012-12-09 12:13:00,604::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-384::DEBUG::2012-12-09 12:13:00,604::task::1172::TaskManager.Task::(prepare) Task=`222dbfc4-bac5-4a37-baf3-f3e2726551aa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-384::DEBUG::2012-12-09 12:13:00,605::task::588::TaskManager.Task::(_updateState) Task=`222dbfc4-bac5-4a37-baf3-f3e2726551aa`::moving from state preparing -> state finished
>Thread-384::DEBUG::2012-12-09 12:13:00,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-384::DEBUG::2012-12-09 12:13:00,606::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-384::DEBUG::2012-12-09 12:13:00,606::task::978::TaskManager.Task::(_decref) Task=`222dbfc4-bac5-4a37-baf3-f3e2726551aa`::ref 0 aborting False
>Thread-385::DEBUG::2012-12-09 12:13:00,625::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-385::DEBUG::2012-12-09 12:13:00,625::task::588::TaskManager.Task::(_updateState) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::moving from state init -> state preparing
>Thread-385::INFO::2012-12-09 12:13:00,625::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-385::DEBUG::2012-12-09 12:13:00,626::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`da43a9d0-ba76-44ab-89ec-cf7f02931cf8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-385::DEBUG::2012-12-09 12:13:00,626::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-385::DEBUG::2012-12-09 12:13:00,626::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-385::DEBUG::2012-12-09 12:13:00,626::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`da43a9d0-ba76-44ab-89ec-cf7f02931cf8`::Granted request
>Thread-385::DEBUG::2012-12-09 12:13:00,627::task::817::TaskManager.Task::(resourceAcquired) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-385::DEBUG::2012-12-09 12:13:00,627::task::978::TaskManager.Task::(_decref) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::ref 1 aborting False
>Thread-385::DEBUG::2012-12-09 12:13:00,628::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-385::DEBUG::2012-12-09 12:13:00,628::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-385::DEBUG::2012-12-09 12:13:00,735::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-385::WARNING::2012-12-09 12:13:00,737::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-385::DEBUG::2012-12-09 12:13:00,738::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-385::WARNING::2012-12-09 12:13:00,739::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-385::INFO::2012-12-09 12:13:00,740::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-385::DEBUG::2012-12-09 12:13:00,740::task::1172::TaskManager.Task::(prepare) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-385::DEBUG::2012-12-09 12:13:00,740::task::588::TaskManager.Task::(_updateState) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::moving from state preparing -> state finished
>Thread-385::DEBUG::2012-12-09 12:13:00,740::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-385::DEBUG::2012-12-09 12:13:00,740::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-385::DEBUG::2012-12-09 12:13:00,741::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-385::DEBUG::2012-12-09 12:13:00,741::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-385::DEBUG::2012-12-09 12:13:00,741::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-385::DEBUG::2012-12-09 12:13:00,741::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-385::DEBUG::2012-12-09 12:13:00,741::task::978::TaskManager.Task::(_decref) Task=`3bfe30f3-4a9e-41ac-bc48-7ff16eb7c32b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:00,912::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:00,964::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265001 s, 38.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:02,970::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:03,017::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227014 s, 45.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:05,024::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:05,066::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208444 s, 49.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:05,189::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:05,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000949529 s, 4.3 MB/s\n'; <rc> = 0
>Thread-388::DEBUG::2012-12-09 12:13:05,617::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-388::DEBUG::2012-12-09 12:13:05,618::task::588::TaskManager.Task::(_updateState) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::moving from state init -> state preparing
>Thread-388::INFO::2012-12-09 12:13:05,619::logUtils::37::dispatcher::(wrapper) Run and protect: getIsoList(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', extension='iso', options=None)
>Thread-388::DEBUG::2012-12-09 12:13:05,619::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`214140c1-b253-4299-9c52-109147dd33f0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-388::DEBUG::2012-12-09 12:13:05,620::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-388::DEBUG::2012-12-09 12:13:05,620::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-388::DEBUG::2012-12-09 12:13:05,621::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`214140c1-b253-4299-9c52-109147dd33f0`::Granted request
>Thread-388::DEBUG::2012-12-09 12:13:05,621::task::817::TaskManager.Task::(resourceAcquired) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-388::DEBUG::2012-12-09 12:13:05,622::task::978::TaskManager.Task::(_decref) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::ref 1 aborting False
>Thread-388::DEBUG::2012-12-09 12:13:05,623::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-388::DEBUG::2012-12-09 12:13:05,624::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-388::DEBUG::2012-12-09 12:13:05,740::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-388::WARNING::2012-12-09 12:13:05,742::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-388::DEBUG::2012-12-09 12:13:05,742::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:05,744::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-388::WARNING::2012-12-09 12:13:05,744::sp::1487::Storage.StoragePool::(getIsoDomain) Storage domain c080b7a1-1d5f-449f-9a65-375961d37141 does not exist
>Thread-51::DEBUG::2012-12-09 12:13:05,746::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-388::ERROR::2012-12-09 12:13:05,747::task::853::TaskManager.Task::(_setError) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 2037, in getIsoList
>    raise se.GetIsoListError(spUUID)
>GetIsoListError: Cannot get ISO list: ('4372535e-3f4d-11e2-8ba0-525400202852',)
>Thread-388::DEBUG::2012-12-09 12:13:05,756::task::872::TaskManager.Task::(_run) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::Task._run: dec21b2d-f656-4fdc-a33b-a6b7be6fad38 ('4372535e-3f4d-11e2-8ba0-525400202852', 'iso') {} failed - stopping task
>Thread-388::DEBUG::2012-12-09 12:13:05,758::task::1199::TaskManager.Task::(stop) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::stopping in state preparing (force False)
>Thread-388::DEBUG::2012-12-09 12:13:05,758::task::978::TaskManager.Task::(_decref) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::ref 1 aborting True
>Thread-388::INFO::2012-12-09 12:13:05,759::task::1157::TaskManager.Task::(prepare) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::aborting: Task is aborted: 'Cannot get ISO list' - code 314
>Thread-388::DEBUG::2012-12-09 12:13:05,759::task::1162::TaskManager.Task::(prepare) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::Prepare: aborted: Cannot get ISO list
>Thread-388::DEBUG::2012-12-09 12:13:05,760::task::978::TaskManager.Task::(_decref) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::ref 0 aborting True
>Thread-388::DEBUG::2012-12-09 12:13:05,760::task::913::TaskManager.Task::(_doAbort) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::Task._doAbort: force False
>Thread-388::DEBUG::2012-12-09 12:13:05,760::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-388::DEBUG::2012-12-09 12:13:05,760::task::588::TaskManager.Task::(_updateState) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::moving from state preparing -> state aborting
>Thread-388::DEBUG::2012-12-09 12:13:05,760::task::537::TaskManager.Task::(__state_aborting) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::_aborting: recover policy none
>Thread-388::DEBUG::2012-12-09 12:13:05,761::task::588::TaskManager.Task::(_updateState) Task=`dec21b2d-f656-4fdc-a33b-a6b7be6fad38`::moving from state aborting -> state failed
>Thread-388::DEBUG::2012-12-09 12:13:05,761::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-388::DEBUG::2012-12-09 12:13:05,761::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-388::DEBUG::2012-12-09 12:13:05,761::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-388::DEBUG::2012-12-09 12:13:05,762::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-388::DEBUG::2012-12-09 12:13:05,762::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-388::DEBUG::2012-12-09 12:13:05,762::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-388::ERROR::2012-12-09 12:13:05,762::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot get ISO list: ('4372535e-3f4d-11e2-8ba0-525400202852',)", 'code': 314}}
>Thread-51::DEBUG::2012-12-09 12:13:05,852::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:05,853::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:05,854::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:05,854::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:07,073::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:07,124::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258685 s, 39.6 MB/s\n'; <rc> = 0
>Thread-390::DEBUG::2012-12-09 12:13:08,112::task::588::TaskManager.Task::(_updateState) Task=`3dcce2a0-b203-44d0-8f1e-fbc4dac4f758`::moving from state init -> state preparing
>Thread-390::INFO::2012-12-09 12:13:08,112::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-390::INFO::2012-12-09 12:13:08,113::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0275001525879', 'lastCheck': '2.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-390::DEBUG::2012-12-09 12:13:08,113::task::1172::TaskManager.Task::(prepare) Task=`3dcce2a0-b203-44d0-8f1e-fbc4dac4f758`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0275001525879', 'lastCheck': '2.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-390::DEBUG::2012-12-09 12:13:08,114::task::588::TaskManager.Task::(_updateState) Task=`3dcce2a0-b203-44d0-8f1e-fbc4dac4f758`::moving from state preparing -> state finished
>Thread-390::DEBUG::2012-12-09 12:13:08,114::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-390::DEBUG::2012-12-09 12:13:08,115::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-390::DEBUG::2012-12-09 12:13:08,115::task::978::TaskManager.Task::(_decref) Task=`3dcce2a0-b203-44d0-8f1e-fbc4dac4f758`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:09,129::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:09,177::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229647 s, 44.6 MB/s\n'; <rc> = 0
>Thread-393::DEBUG::2012-12-09 12:13:10,869::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-393::DEBUG::2012-12-09 12:13:10,870::task::588::TaskManager.Task::(_updateState) Task=`4e1bb5d4-68c0-498c-8554-ef1be1daa046`::moving from state init -> state preparing
>Thread-393::INFO::2012-12-09 12:13:10,870::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-393::INFO::2012-12-09 12:13:10,871::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-393::DEBUG::2012-12-09 12:13:10,871::task::1172::TaskManager.Task::(prepare) Task=`4e1bb5d4-68c0-498c-8554-ef1be1daa046`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-393::DEBUG::2012-12-09 12:13:10,871::task::588::TaskManager.Task::(_updateState) Task=`4e1bb5d4-68c0-498c-8554-ef1be1daa046`::moving from state preparing -> state finished
>Thread-393::DEBUG::2012-12-09 12:13:10,871::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-393::DEBUG::2012-12-09 12:13:10,871::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-393::DEBUG::2012-12-09 12:13:10,871::task::978::TaskManager.Task::(_decref) Task=`4e1bb5d4-68c0-498c-8554-ef1be1daa046`::ref 0 aborting False
>Thread-394::DEBUG::2012-12-09 12:13:10,887::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-394::DEBUG::2012-12-09 12:13:10,888::task::588::TaskManager.Task::(_updateState) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::moving from state init -> state preparing
>Thread-394::INFO::2012-12-09 12:13:10,888::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-394::DEBUG::2012-12-09 12:13:10,889::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d74a086-5ff6-4e98-a264-ceaeb05ab447`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-394::DEBUG::2012-12-09 12:13:10,889::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-394::DEBUG::2012-12-09 12:13:10,890::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-394::DEBUG::2012-12-09 12:13:10,890::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d74a086-5ff6-4e98-a264-ceaeb05ab447`::Granted request
>Thread-394::DEBUG::2012-12-09 12:13:10,891::task::817::TaskManager.Task::(resourceAcquired) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-394::DEBUG::2012-12-09 12:13:10,892::task::978::TaskManager.Task::(_decref) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::ref 1 aborting False
>Thread-394::DEBUG::2012-12-09 12:13:10,893::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-394::DEBUG::2012-12-09 12:13:10,894::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-394::DEBUG::2012-12-09 12:13:11,011::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-394::WARNING::2012-12-09 12:13:11,013::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-394::DEBUG::2012-12-09 12:13:11,013::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-394::WARNING::2012-12-09 12:13:11,014::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-394::INFO::2012-12-09 12:13:11,015::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-394::DEBUG::2012-12-09 12:13:11,016::task::1172::TaskManager.Task::(prepare) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-394::DEBUG::2012-12-09 12:13:11,016::task::588::TaskManager.Task::(_updateState) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::moving from state preparing -> state finished
>Thread-394::DEBUG::2012-12-09 12:13:11,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-394::DEBUG::2012-12-09 12:13:11,017::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-394::DEBUG::2012-12-09 12:13:11,018::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-394::DEBUG::2012-12-09 12:13:11,018::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-394::DEBUG::2012-12-09 12:13:11,019::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-394::DEBUG::2012-12-09 12:13:11,019::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-394::DEBUG::2012-12-09 12:13:11,019::task::978::TaskManager.Task::(_decref) Task=`f4ee7da5-71b5-473c-8885-8f906da3b57c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:11,183::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:11,238::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277526 s, 36.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:13,245::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:13,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022028 s, 46.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:15,219::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:15,235::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000827371 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:15,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:15,353::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285115 s, 35.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:13:15,855::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:15,856::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:13:15,972::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:15,974::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:15,974::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:15,976::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:17,359::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:17,406::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217225 s, 47.1 MB/s\n'; <rc> = 0
>Thread-398::DEBUG::2012-12-09 12:13:18,391::task::588::TaskManager.Task::(_updateState) Task=`5f3e9460-7e22-4751-8d30-c51236534784`::moving from state init -> state preparing
>Thread-398::INFO::2012-12-09 12:13:18,391::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-398::INFO::2012-12-09 12:13:18,392::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168879032135', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-398::DEBUG::2012-12-09 12:13:18,392::task::1172::TaskManager.Task::(prepare) Task=`5f3e9460-7e22-4751-8d30-c51236534784`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168879032135', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-398::DEBUG::2012-12-09 12:13:18,392::task::588::TaskManager.Task::(_updateState) Task=`5f3e9460-7e22-4751-8d30-c51236534784`::moving from state preparing -> state finished
>Thread-398::DEBUG::2012-12-09 12:13:18,392::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-398::DEBUG::2012-12-09 12:13:18,392::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-398::DEBUG::2012-12-09 12:13:18,393::task::978::TaskManager.Task::(_decref) Task=`5f3e9460-7e22-4751-8d30-c51236534784`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:19,413::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:19,458::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201284 s, 50.9 MB/s\n'; <rc> = 0
>Thread-401::DEBUG::2012-12-09 12:13:21,102::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-401::DEBUG::2012-12-09 12:13:21,103::task::588::TaskManager.Task::(_updateState) Task=`57da7823-da11-4d41-a381-aa1cee789c90`::moving from state init -> state preparing
>Thread-401::INFO::2012-12-09 12:13:21,103::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-401::INFO::2012-12-09 12:13:21,104::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-401::DEBUG::2012-12-09 12:13:21,104::task::1172::TaskManager.Task::(prepare) Task=`57da7823-da11-4d41-a381-aa1cee789c90`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-401::DEBUG::2012-12-09 12:13:21,105::task::588::TaskManager.Task::(_updateState) Task=`57da7823-da11-4d41-a381-aa1cee789c90`::moving from state preparing -> state finished
>Thread-401::DEBUG::2012-12-09 12:13:21,105::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-401::DEBUG::2012-12-09 12:13:21,106::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-401::DEBUG::2012-12-09 12:13:21,106::task::978::TaskManager.Task::(_decref) Task=`57da7823-da11-4d41-a381-aa1cee789c90`::ref 0 aborting False
>Thread-402::DEBUG::2012-12-09 12:13:21,131::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-402::DEBUG::2012-12-09 12:13:21,132::task::588::TaskManager.Task::(_updateState) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::moving from state init -> state preparing
>Thread-402::INFO::2012-12-09 12:13:21,132::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-402::DEBUG::2012-12-09 12:13:21,133::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`755771f8-736e-4e6b-b394-de29f712724a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-402::DEBUG::2012-12-09 12:13:21,133::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-402::DEBUG::2012-12-09 12:13:21,134::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-402::DEBUG::2012-12-09 12:13:21,134::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`755771f8-736e-4e6b-b394-de29f712724a`::Granted request
>Thread-402::DEBUG::2012-12-09 12:13:21,135::task::817::TaskManager.Task::(resourceAcquired) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-402::DEBUG::2012-12-09 12:13:21,135::task::978::TaskManager.Task::(_decref) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::ref 1 aborting False
>Thread-402::DEBUG::2012-12-09 12:13:21,137::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-402::DEBUG::2012-12-09 12:13:21,138::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-402::DEBUG::2012-12-09 12:13:21,249::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-402::WARNING::2012-12-09 12:13:21,251::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-402::DEBUG::2012-12-09 12:13:21,251::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-402::WARNING::2012-12-09 12:13:21,252::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-402::INFO::2012-12-09 12:13:21,253::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-402::DEBUG::2012-12-09 12:13:21,253::task::1172::TaskManager.Task::(prepare) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-402::DEBUG::2012-12-09 12:13:21,254::task::588::TaskManager.Task::(_updateState) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::moving from state preparing -> state finished
>Thread-402::DEBUG::2012-12-09 12:13:21,254::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-402::DEBUG::2012-12-09 12:13:21,255::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-402::DEBUG::2012-12-09 12:13:21,255::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-402::DEBUG::2012-12-09 12:13:21,256::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-402::DEBUG::2012-12-09 12:13:21,256::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-402::DEBUG::2012-12-09 12:13:21,256::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-402::DEBUG::2012-12-09 12:13:21,256::task::978::TaskManager.Task::(_decref) Task=`ccfe7454-caf7-4fe2-8a3a-c2c3c916df77`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:21,464::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:21,509::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190808 s, 53.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:23,516::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:23,564::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250499 s, 40.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:25,241::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:25,253::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000771184 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:25,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:25,623::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196019 s, 52.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:13:25,977::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:25,978::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:13:26,090::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:26,091::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:26,091::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:26,092::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:27,630::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:27,683::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281141 s, 36.4 MB/s\n'; <rc> = 0
>Thread-406::DEBUG::2012-12-09 12:13:28,678::task::588::TaskManager.Task::(_updateState) Task=`40cd8570-7660-431b-b6b9-4b577a2ed62b`::moving from state init -> state preparing
>Thread-406::INFO::2012-12-09 12:13:28,679::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-406::INFO::2012-12-09 12:13:28,680::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135569572449', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-406::DEBUG::2012-12-09 12:13:28,680::task::1172::TaskManager.Task::(prepare) Task=`40cd8570-7660-431b-b6b9-4b577a2ed62b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135569572449', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-406::DEBUG::2012-12-09 12:13:28,681::task::588::TaskManager.Task::(_updateState) Task=`40cd8570-7660-431b-b6b9-4b577a2ed62b`::moving from state preparing -> state finished
>Thread-406::DEBUG::2012-12-09 12:13:28,681::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-406::DEBUG::2012-12-09 12:13:28,681::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-406::DEBUG::2012-12-09 12:13:28,682::task::978::TaskManager.Task::(_decref) Task=`40cd8570-7660-431b-b6b9-4b577a2ed62b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:29,690::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:29,744::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282393 s, 36.3 MB/s\n'; <rc> = 0
>Thread-409::DEBUG::2012-12-09 12:13:31,333::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-409::DEBUG::2012-12-09 12:13:31,334::task::588::TaskManager.Task::(_updateState) Task=`489827a2-1562-4a0a-859f-490ee72ccee1`::moving from state init -> state preparing
>Thread-409::INFO::2012-12-09 12:13:31,335::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-409::INFO::2012-12-09 12:13:31,335::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-409::DEBUG::2012-12-09 12:13:31,336::task::1172::TaskManager.Task::(prepare) Task=`489827a2-1562-4a0a-859f-490ee72ccee1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-409::DEBUG::2012-12-09 12:13:31,336::task::588::TaskManager.Task::(_updateState) Task=`489827a2-1562-4a0a-859f-490ee72ccee1`::moving from state preparing -> state finished
>Thread-409::DEBUG::2012-12-09 12:13:31,337::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-409::DEBUG::2012-12-09 12:13:31,337::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-409::DEBUG::2012-12-09 12:13:31,337::task::978::TaskManager.Task::(_decref) Task=`489827a2-1562-4a0a-859f-490ee72ccee1`::ref 0 aborting False
>Thread-410::DEBUG::2012-12-09 12:13:31,362::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-410::DEBUG::2012-12-09 12:13:31,363::task::588::TaskManager.Task::(_updateState) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::moving from state init -> state preparing
>Thread-410::INFO::2012-12-09 12:13:31,363::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-410::DEBUG::2012-12-09 12:13:31,364::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd5555e2-430d-4961-aa58-6d34a6f597cf`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-410::DEBUG::2012-12-09 12:13:31,364::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-410::DEBUG::2012-12-09 12:13:31,365::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-410::DEBUG::2012-12-09 12:13:31,365::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd5555e2-430d-4961-aa58-6d34a6f597cf`::Granted request
>Thread-410::DEBUG::2012-12-09 12:13:31,366::task::817::TaskManager.Task::(resourceAcquired) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-410::DEBUG::2012-12-09 12:13:31,366::task::978::TaskManager.Task::(_decref) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::ref 1 aborting False
>Thread-410::DEBUG::2012-12-09 12:13:31,368::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-410::DEBUG::2012-12-09 12:13:31,369::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-410::DEBUG::2012-12-09 12:13:31,470::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-410::WARNING::2012-12-09 12:13:31,472::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-410::DEBUG::2012-12-09 12:13:31,472::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-410::WARNING::2012-12-09 12:13:31,473::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-410::INFO::2012-12-09 12:13:31,474::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-410::DEBUG::2012-12-09 12:13:31,474::task::1172::TaskManager.Task::(prepare) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-410::DEBUG::2012-12-09 12:13:31,474::task::588::TaskManager.Task::(_updateState) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::moving from state preparing -> state finished
>Thread-410::DEBUG::2012-12-09 12:13:31,475::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-410::DEBUG::2012-12-09 12:13:31,475::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-410::DEBUG::2012-12-09 12:13:31,475::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-410::DEBUG::2012-12-09 12:13:31,476::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-410::DEBUG::2012-12-09 12:13:31,476::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-410::DEBUG::2012-12-09 12:13:31,476::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-410::DEBUG::2012-12-09 12:13:31,477::task::978::TaskManager.Task::(_decref) Task=`cd0eeaf5-37d0-4674-b993-fc85677d86d3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:31,750::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:31,795::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208373 s, 49.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:33,801::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:33,846::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221614 s, 46.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:35,259::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:35,275::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000716525 s, 5.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:35,853::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:35,898::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205558 s, 49.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:13:36,093::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:36,094::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:13:36,207::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:36,208::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:36,209::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:36,209::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:37,905::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:37,953::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248334 s, 41.2 MB/s\n'; <rc> = 0
>Thread-414::DEBUG::2012-12-09 12:13:38,978::task::588::TaskManager.Task::(_updateState) Task=`0da96eeb-185b-4cb2-9b57-634a1886806b`::moving from state init -> state preparing
>Thread-414::INFO::2012-12-09 12:13:38,978::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-414::INFO::2012-12-09 12:13:38,978::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169670581818', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-414::DEBUG::2012-12-09 12:13:38,978::task::1172::TaskManager.Task::(prepare) Task=`0da96eeb-185b-4cb2-9b57-634a1886806b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169670581818', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-414::DEBUG::2012-12-09 12:13:38,979::task::588::TaskManager.Task::(_updateState) Task=`0da96eeb-185b-4cb2-9b57-634a1886806b`::moving from state preparing -> state finished
>Thread-414::DEBUG::2012-12-09 12:13:38,979::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-414::DEBUG::2012-12-09 12:13:38,979::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-414::DEBUG::2012-12-09 12:13:38,979::task::978::TaskManager.Task::(_decref) Task=`0da96eeb-185b-4cb2-9b57-634a1886806b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:39,960::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:40,014::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280748 s, 36.5 MB/s\n'; <rc> = 0
>Thread-417::DEBUG::2012-12-09 12:13:41,614::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-417::DEBUG::2012-12-09 12:13:41,615::task::588::TaskManager.Task::(_updateState) Task=`d1666b6a-d91a-4086-87a4-1cd882883833`::moving from state init -> state preparing
>Thread-417::INFO::2012-12-09 12:13:41,615::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-417::INFO::2012-12-09 12:13:41,616::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-417::DEBUG::2012-12-09 12:13:41,616::task::1172::TaskManager.Task::(prepare) Task=`d1666b6a-d91a-4086-87a4-1cd882883833`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-417::DEBUG::2012-12-09 12:13:41,616::task::588::TaskManager.Task::(_updateState) Task=`d1666b6a-d91a-4086-87a4-1cd882883833`::moving from state preparing -> state finished
>Thread-417::DEBUG::2012-12-09 12:13:41,616::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-417::DEBUG::2012-12-09 12:13:41,616::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-417::DEBUG::2012-12-09 12:13:41,617::task::978::TaskManager.Task::(_decref) Task=`d1666b6a-d91a-4086-87a4-1cd882883833`::ref 0 aborting False
>Thread-418::DEBUG::2012-12-09 12:13:41,631::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-418::DEBUG::2012-12-09 12:13:41,632::task::588::TaskManager.Task::(_updateState) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::moving from state init -> state preparing
>Thread-418::INFO::2012-12-09 12:13:41,633::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-418::DEBUG::2012-12-09 12:13:41,633::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0d4d4ad5-1300-4788-9167-4ac9668e0648`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-418::DEBUG::2012-12-09 12:13:41,634::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-418::DEBUG::2012-12-09 12:13:41,634::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-418::DEBUG::2012-12-09 12:13:41,635::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0d4d4ad5-1300-4788-9167-4ac9668e0648`::Granted request
>Thread-418::DEBUG::2012-12-09 12:13:41,636::task::817::TaskManager.Task::(resourceAcquired) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-418::DEBUG::2012-12-09 12:13:41,636::task::978::TaskManager.Task::(_decref) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::ref 1 aborting False
>Thread-418::DEBUG::2012-12-09 12:13:41,638::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-418::DEBUG::2012-12-09 12:13:41,638::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-418::DEBUG::2012-12-09 12:13:41,750::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-418::WARNING::2012-12-09 12:13:41,751::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-418::DEBUG::2012-12-09 12:13:41,751::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-418::WARNING::2012-12-09 12:13:41,752::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-418::INFO::2012-12-09 12:13:41,752::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-418::DEBUG::2012-12-09 12:13:41,752::task::1172::TaskManager.Task::(prepare) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-418::DEBUG::2012-12-09 12:13:41,752::task::588::TaskManager.Task::(_updateState) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::moving from state preparing -> state finished
>Thread-418::DEBUG::2012-12-09 12:13:41,753::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-418::DEBUG::2012-12-09 12:13:41,753::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-418::DEBUG::2012-12-09 12:13:41,753::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-418::DEBUG::2012-12-09 12:13:41,753::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-418::DEBUG::2012-12-09 12:13:41,753::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-418::DEBUG::2012-12-09 12:13:41,754::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-418::DEBUG::2012-12-09 12:13:41,754::task::978::TaskManager.Task::(_decref) Task=`123e2fac-b0b7-4ac0-905e-4aa5a77f5ab6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:42,020::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:42,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187442 s, 54.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:44,071::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:44,117::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215383 s, 47.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:45,281::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:45,296::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000921412 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:46,124::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:46,174::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025396 s, 40.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:13:46,210::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:46,210::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:13:46,320::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:46,321::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:46,322::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:46,322::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:48,180::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:48,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160577 s, 63.8 MB/s\n'; <rc> = 0
>Thread-422::DEBUG::2012-12-09 12:13:49,254::task::588::TaskManager.Task::(_updateState) Task=`e641abbf-b473-4375-902d-10a06ecb29ae`::moving from state init -> state preparing
>Thread-422::INFO::2012-12-09 12:13:49,255::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-422::INFO::2012-12-09 12:13:49,256::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156741142273', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-422::DEBUG::2012-12-09 12:13:49,256::task::1172::TaskManager.Task::(prepare) Task=`e641abbf-b473-4375-902d-10a06ecb29ae`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156741142273', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-422::DEBUG::2012-12-09 12:13:49,256::task::588::TaskManager.Task::(_updateState) Task=`e641abbf-b473-4375-902d-10a06ecb29ae`::moving from state preparing -> state finished
>Thread-422::DEBUG::2012-12-09 12:13:49,257::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-422::DEBUG::2012-12-09 12:13:49,257::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-422::DEBUG::2012-12-09 12:13:49,258::task::978::TaskManager.Task::(_decref) Task=`e641abbf-b473-4375-902d-10a06ecb29ae`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:50,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:50,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245934 s, 41.6 MB/s\n'; <rc> = 0
>Thread-425::DEBUG::2012-12-09 12:13:51,950::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-425::DEBUG::2012-12-09 12:13:51,951::task::588::TaskManager.Task::(_updateState) Task=`ba6780c7-3d58-4906-abcf-5c471e2444e1`::moving from state init -> state preparing
>Thread-425::INFO::2012-12-09 12:13:51,952::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-425::INFO::2012-12-09 12:13:51,952::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-425::DEBUG::2012-12-09 12:13:51,953::task::1172::TaskManager.Task::(prepare) Task=`ba6780c7-3d58-4906-abcf-5c471e2444e1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-425::DEBUG::2012-12-09 12:13:51,953::task::588::TaskManager.Task::(_updateState) Task=`ba6780c7-3d58-4906-abcf-5c471e2444e1`::moving from state preparing -> state finished
>Thread-425::DEBUG::2012-12-09 12:13:51,953::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-425::DEBUG::2012-12-09 12:13:51,954::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-425::DEBUG::2012-12-09 12:13:51,954::task::978::TaskManager.Task::(_decref) Task=`ba6780c7-3d58-4906-abcf-5c471e2444e1`::ref 0 aborting False
>Thread-426::DEBUG::2012-12-09 12:13:51,980::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-426::DEBUG::2012-12-09 12:13:51,981::task::588::TaskManager.Task::(_updateState) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::moving from state init -> state preparing
>Thread-426::INFO::2012-12-09 12:13:51,982::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-426::DEBUG::2012-12-09 12:13:51,982::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`01be9c5e-60ac-46d0-9251-b673b1ca333e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-426::DEBUG::2012-12-09 12:13:51,983::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-426::DEBUG::2012-12-09 12:13:51,983::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-426::DEBUG::2012-12-09 12:13:51,984::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`01be9c5e-60ac-46d0-9251-b673b1ca333e`::Granted request
>Thread-426::DEBUG::2012-12-09 12:13:51,985::task::817::TaskManager.Task::(resourceAcquired) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-426::DEBUG::2012-12-09 12:13:51,986::task::978::TaskManager.Task::(_decref) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::ref 1 aborting False
>Thread-426::DEBUG::2012-12-09 12:13:51,987::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-426::DEBUG::2012-12-09 12:13:51,988::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-426::DEBUG::2012-12-09 12:13:52,094::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-426::WARNING::2012-12-09 12:13:52,096::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-426::DEBUG::2012-12-09 12:13:52,097::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-426::WARNING::2012-12-09 12:13:52,098::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-426::INFO::2012-12-09 12:13:52,098::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-426::DEBUG::2012-12-09 12:13:52,099::task::1172::TaskManager.Task::(prepare) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-426::DEBUG::2012-12-09 12:13:52,099::task::588::TaskManager.Task::(_updateState) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::moving from state preparing -> state finished
>Thread-426::DEBUG::2012-12-09 12:13:52,100::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-426::DEBUG::2012-12-09 12:13:52,100::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-426::DEBUG::2012-12-09 12:13:52,100::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-426::DEBUG::2012-12-09 12:13:52,100::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-426::DEBUG::2012-12-09 12:13:52,101::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-426::DEBUG::2012-12-09 12:13:52,101::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-426::DEBUG::2012-12-09 12:13:52,101::task::978::TaskManager.Task::(_decref) Task=`d8e867e9-c775-4de6-81fa-fc586f601776`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:13:52,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:52,336::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273743 s, 37.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:54,343::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:54,388::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204596 s, 50.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:13:55,300::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:13:55,316::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00077442 s, 5.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:13:56,323::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:13:56,324::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:56,394::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:13:56,433::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:13:56,434::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:13:56,434::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:13:56,434::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:13:56,436::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159319 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:13:58,442::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:13:58,493::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261171 s, 39.2 MB/s\n'; <rc> = 0
>Thread-430::DEBUG::2012-12-09 12:13:59,578::task::588::TaskManager.Task::(_updateState) Task=`41625e81-d6e0-4533-ab18-6d52094ce0b7`::moving from state init -> state preparing
>Thread-430::INFO::2012-12-09 12:13:59,579::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-430::INFO::2012-12-09 12:13:59,579::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161130428314', 'lastCheck': '4.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-430::DEBUG::2012-12-09 12:13:59,579::task::1172::TaskManager.Task::(prepare) Task=`41625e81-d6e0-4533-ab18-6d52094ce0b7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161130428314', 'lastCheck': '4.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-430::DEBUG::2012-12-09 12:13:59,579::task::588::TaskManager.Task::(_updateState) Task=`41625e81-d6e0-4533-ab18-6d52094ce0b7`::moving from state preparing -> state finished
>Thread-430::DEBUG::2012-12-09 12:13:59,580::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-430::DEBUG::2012-12-09 12:13:59,580::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-430::DEBUG::2012-12-09 12:13:59,580::task::978::TaskManager.Task::(_decref) Task=`41625e81-d6e0-4533-ab18-6d52094ce0b7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:00,500::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:00,546::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220061 s, 46.5 MB/s\n'; <rc> = 0
>Thread-433::DEBUG::2012-12-09 12:14:02,271::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-433::DEBUG::2012-12-09 12:14:02,272::task::588::TaskManager.Task::(_updateState) Task=`2ad24681-52e6-4afd-9ef9-d10054a39d00`::moving from state init -> state preparing
>Thread-433::INFO::2012-12-09 12:14:02,273::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-433::INFO::2012-12-09 12:14:02,274::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-433::DEBUG::2012-12-09 12:14:02,274::task::1172::TaskManager.Task::(prepare) Task=`2ad24681-52e6-4afd-9ef9-d10054a39d00`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-433::DEBUG::2012-12-09 12:14:02,274::task::588::TaskManager.Task::(_updateState) Task=`2ad24681-52e6-4afd-9ef9-d10054a39d00`::moving from state preparing -> state finished
>Thread-433::DEBUG::2012-12-09 12:14:02,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-433::DEBUG::2012-12-09 12:14:02,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-433::DEBUG::2012-12-09 12:14:02,276::task::978::TaskManager.Task::(_decref) Task=`2ad24681-52e6-4afd-9ef9-d10054a39d00`::ref 0 aborting False
>Thread-434::DEBUG::2012-12-09 12:14:02,301::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-434::DEBUG::2012-12-09 12:14:02,302::task::588::TaskManager.Task::(_updateState) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::moving from state init -> state preparing
>Thread-434::INFO::2012-12-09 12:14:02,302::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-434::DEBUG::2012-12-09 12:14:02,303::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`531713ef-26b6-4487-8e5c-e099b0313c28`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-434::DEBUG::2012-12-09 12:14:02,303::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-434::DEBUG::2012-12-09 12:14:02,304::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-434::DEBUG::2012-12-09 12:14:02,304::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`531713ef-26b6-4487-8e5c-e099b0313c28`::Granted request
>Thread-434::DEBUG::2012-12-09 12:14:02,305::task::817::TaskManager.Task::(resourceAcquired) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-434::DEBUG::2012-12-09 12:14:02,305::task::978::TaskManager.Task::(_decref) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::ref 1 aborting False
>Thread-434::DEBUG::2012-12-09 12:14:02,307::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-434::DEBUG::2012-12-09 12:14:02,308::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-434::DEBUG::2012-12-09 12:14:02,416::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-434::WARNING::2012-12-09 12:14:02,418::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-434::DEBUG::2012-12-09 12:14:02,418::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-434::WARNING::2012-12-09 12:14:02,420::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-434::INFO::2012-12-09 12:14:02,420::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-434::DEBUG::2012-12-09 12:14:02,421::task::1172::TaskManager.Task::(prepare) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-434::DEBUG::2012-12-09 12:14:02,421::task::588::TaskManager.Task::(_updateState) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::moving from state preparing -> state finished
>Thread-434::DEBUG::2012-12-09 12:14:02,422::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-434::DEBUG::2012-12-09 12:14:02,422::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-434::DEBUG::2012-12-09 12:14:02,423::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-434::DEBUG::2012-12-09 12:14:02,423::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-434::DEBUG::2012-12-09 12:14:02,424::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-434::DEBUG::2012-12-09 12:14:02,424::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-434::DEBUG::2012-12-09 12:14:02,425::task::978::TaskManager.Task::(_decref) Task=`85a03492-36f5-4386-8a5c-2fb3237995f8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:02,553::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:02,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266916 s, 38.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:04,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:04,663::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265043 s, 38.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:05,319::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:14:05,320::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:14:05,320::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:05,327::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000512085 s, 4.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:05,327::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000512085 s, 4.0 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:14:05,328::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:14:05,328::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:14:05,328::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:14:05,328::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,329::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,329::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,329::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,329::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,330::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:05,438::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:05,440::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:14:05,440::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:14:05,441::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:14:05,441::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:14:05,441::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:14:05,441::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:05,556::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:05,557::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:05,568::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000924196 s, 4.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:06,435::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:14:06,436::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:06,437::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:06,552::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:06,554::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:06,555::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:06,556::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:06,669::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:06,716::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220132 s, 46.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:08,723::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:08,774::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265281 s, 38.6 MB/s\n'; <rc> = 0
>Thread-438::DEBUG::2012-12-09 12:14:09,938::task::588::TaskManager.Task::(_updateState) Task=`6d70ebf1-6731-4756-a871-7c0baa8581fd`::moving from state init -> state preparing
>Thread-438::INFO::2012-12-09 12:14:09,938::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-438::INFO::2012-12-09 12:14:09,939::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011528968811', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-438::DEBUG::2012-12-09 12:14:09,939::task::1172::TaskManager.Task::(prepare) Task=`6d70ebf1-6731-4756-a871-7c0baa8581fd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011528968811', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-438::DEBUG::2012-12-09 12:14:09,940::task::588::TaskManager.Task::(_updateState) Task=`6d70ebf1-6731-4756-a871-7c0baa8581fd`::moving from state preparing -> state finished
>Thread-438::DEBUG::2012-12-09 12:14:09,940::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-438::DEBUG::2012-12-09 12:14:09,941::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-438::DEBUG::2012-12-09 12:14:09,941::task::978::TaskManager.Task::(_decref) Task=`6d70ebf1-6731-4756-a871-7c0baa8581fd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:10,780::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:10,827::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212669 s, 48.1 MB/s\n'; <rc> = 0
>Thread-441::DEBUG::2012-12-09 12:14:12,532::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-441::DEBUG::2012-12-09 12:14:12,533::task::588::TaskManager.Task::(_updateState) Task=`0adce498-49d8-4eec-a035-f69e144abffa`::moving from state init -> state preparing
>Thread-441::INFO::2012-12-09 12:14:12,534::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-441::INFO::2012-12-09 12:14:12,535::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-441::DEBUG::2012-12-09 12:14:12,535::task::1172::TaskManager.Task::(prepare) Task=`0adce498-49d8-4eec-a035-f69e144abffa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-441::DEBUG::2012-12-09 12:14:12,535::task::588::TaskManager.Task::(_updateState) Task=`0adce498-49d8-4eec-a035-f69e144abffa`::moving from state preparing -> state finished
>Thread-441::DEBUG::2012-12-09 12:14:12,536::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-441::DEBUG::2012-12-09 12:14:12,536::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-441::DEBUG::2012-12-09 12:14:12,537::task::978::TaskManager.Task::(_decref) Task=`0adce498-49d8-4eec-a035-f69e144abffa`::ref 0 aborting False
>Thread-442::DEBUG::2012-12-09 12:14:12,562::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-442::DEBUG::2012-12-09 12:14:12,563::task::588::TaskManager.Task::(_updateState) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::moving from state init -> state preparing
>Thread-442::INFO::2012-12-09 12:14:12,564::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-442::DEBUG::2012-12-09 12:14:12,564::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`55a7326a-944a-4eb8-b48b-de5eb4c747a8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-442::DEBUG::2012-12-09 12:14:12,565::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-442::DEBUG::2012-12-09 12:14:12,565::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-442::DEBUG::2012-12-09 12:14:12,566::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`55a7326a-944a-4eb8-b48b-de5eb4c747a8`::Granted request
>Thread-442::DEBUG::2012-12-09 12:14:12,566::task::817::TaskManager.Task::(resourceAcquired) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-442::DEBUG::2012-12-09 12:14:12,567::task::978::TaskManager.Task::(_decref) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::ref 1 aborting False
>Thread-442::DEBUG::2012-12-09 12:14:12,568::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-442::DEBUG::2012-12-09 12:14:12,569::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-442::DEBUG::2012-12-09 12:14:12,679::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-442::WARNING::2012-12-09 12:14:12,680::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-442::DEBUG::2012-12-09 12:14:12,681::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-442::WARNING::2012-12-09 12:14:12,681::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-442::INFO::2012-12-09 12:14:12,681::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-442::DEBUG::2012-12-09 12:14:12,682::task::1172::TaskManager.Task::(prepare) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-442::DEBUG::2012-12-09 12:14:12,682::task::588::TaskManager.Task::(_updateState) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::moving from state preparing -> state finished
>Thread-442::DEBUG::2012-12-09 12:14:12,682::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-442::DEBUG::2012-12-09 12:14:12,682::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-442::DEBUG::2012-12-09 12:14:12,683::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-442::DEBUG::2012-12-09 12:14:12,683::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-442::DEBUG::2012-12-09 12:14:12,683::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-442::DEBUG::2012-12-09 12:14:12,683::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-442::DEBUG::2012-12-09 12:14:12,683::task::978::TaskManager.Task::(_decref) Task=`2d14c211-1fd0-4def-83dd-f22b528e3bfd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:12,834::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:12,884::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257896 s, 39.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:14,891::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:14,938::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220941 s, 46.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:15,572::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:15,587::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000798406 s, 5.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:16,557::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:16,558::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:16,669::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:16,670::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:16,671::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:16,671::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:16,945::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:17,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249893 s, 41.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:19,008::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:19,056::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234084 s, 43.7 MB/s\n'; <rc> = 0
>Thread-446::DEBUG::2012-12-09 12:14:20,299::task::588::TaskManager.Task::(_updateState) Task=`4ed12885-78b8-4036-96d9-17484d8e10ab`::moving from state init -> state preparing
>Thread-446::INFO::2012-12-09 12:14:20,300::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-446::INFO::2012-12-09 12:14:20,300::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.015734910965', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-446::DEBUG::2012-12-09 12:14:20,300::task::1172::TaskManager.Task::(prepare) Task=`4ed12885-78b8-4036-96d9-17484d8e10ab`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.015734910965', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-446::DEBUG::2012-12-09 12:14:20,300::task::588::TaskManager.Task::(_updateState) Task=`4ed12885-78b8-4036-96d9-17484d8e10ab`::moving from state preparing -> state finished
>Thread-446::DEBUG::2012-12-09 12:14:20,301::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-446::DEBUG::2012-12-09 12:14:20,301::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-446::DEBUG::2012-12-09 12:14:20,301::task::978::TaskManager.Task::(_decref) Task=`4ed12885-78b8-4036-96d9-17484d8e10ab`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:21,062::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:21,113::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260293 s, 39.3 MB/s\n'; <rc> = 0
>Thread-449::DEBUG::2012-12-09 12:14:22,814::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-449::DEBUG::2012-12-09 12:14:22,815::task::588::TaskManager.Task::(_updateState) Task=`ef111f07-0dbd-4a8c-b8fd-323a30ac96ea`::moving from state init -> state preparing
>Thread-449::INFO::2012-12-09 12:14:22,815::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-449::INFO::2012-12-09 12:14:22,816::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-449::DEBUG::2012-12-09 12:14:22,816::task::1172::TaskManager.Task::(prepare) Task=`ef111f07-0dbd-4a8c-b8fd-323a30ac96ea`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-449::DEBUG::2012-12-09 12:14:22,817::task::588::TaskManager.Task::(_updateState) Task=`ef111f07-0dbd-4a8c-b8fd-323a30ac96ea`::moving from state preparing -> state finished
>Thread-449::DEBUG::2012-12-09 12:14:22,817::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-449::DEBUG::2012-12-09 12:14:22,818::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-449::DEBUG::2012-12-09 12:14:22,818::task::978::TaskManager.Task::(_decref) Task=`ef111f07-0dbd-4a8c-b8fd-323a30ac96ea`::ref 0 aborting False
>Thread-450::DEBUG::2012-12-09 12:14:22,842::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-450::DEBUG::2012-12-09 12:14:22,843::task::588::TaskManager.Task::(_updateState) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::moving from state init -> state preparing
>Thread-450::INFO::2012-12-09 12:14:22,844::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-450::DEBUG::2012-12-09 12:14:22,844::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c5ee807f-2162-445a-bd1d-ef28ac2d219a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-450::DEBUG::2012-12-09 12:14:22,845::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-450::DEBUG::2012-12-09 12:14:22,845::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-450::DEBUG::2012-12-09 12:14:22,846::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c5ee807f-2162-445a-bd1d-ef28ac2d219a`::Granted request
>Thread-450::DEBUG::2012-12-09 12:14:22,846::task::817::TaskManager.Task::(resourceAcquired) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-450::DEBUG::2012-12-09 12:14:22,847::task::978::TaskManager.Task::(_decref) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::ref 1 aborting False
>Thread-450::DEBUG::2012-12-09 12:14:22,848::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-450::DEBUG::2012-12-09 12:14:22,849::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-450::DEBUG::2012-12-09 12:14:22,959::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-450::WARNING::2012-12-09 12:14:22,960::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-450::DEBUG::2012-12-09 12:14:22,961::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-450::WARNING::2012-12-09 12:14:22,961::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-450::INFO::2012-12-09 12:14:22,961::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-450::DEBUG::2012-12-09 12:14:22,962::task::1172::TaskManager.Task::(prepare) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-450::DEBUG::2012-12-09 12:14:22,962::task::588::TaskManager.Task::(_updateState) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::moving from state preparing -> state finished
>Thread-450::DEBUG::2012-12-09 12:14:22,962::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-450::DEBUG::2012-12-09 12:14:22,962::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-450::DEBUG::2012-12-09 12:14:22,962::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-450::DEBUG::2012-12-09 12:14:22,963::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-450::DEBUG::2012-12-09 12:14:22,963::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-450::DEBUG::2012-12-09 12:14:22,963::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-450::DEBUG::2012-12-09 12:14:22,963::task::978::TaskManager.Task::(_decref) Task=`be460204-d0b1-4396-90f2-09219cc224d2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:23,119::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:23,175::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214711 s, 47.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:25,182::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:25,233::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211201 s, 48.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:25,592::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:25,607::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000856861 s, 4.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:26,672::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:26,673::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:26,780::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:26,781::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:26,782::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:26,782::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:27,239::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:27,286::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220702 s, 46.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:29,293::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:29,343::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235404 s, 43.5 MB/s\n'; <rc> = 0
>Thread-454::DEBUG::2012-12-09 12:14:30,634::task::588::TaskManager.Task::(_updateState) Task=`2861543c-dc4d-4ccf-8a29-006db75c47cb`::moving from state init -> state preparing
>Thread-454::INFO::2012-12-09 12:14:30,635::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-454::INFO::2012-12-09 12:14:30,636::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166409015656', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-454::DEBUG::2012-12-09 12:14:30,636::task::1172::TaskManager.Task::(prepare) Task=`2861543c-dc4d-4ccf-8a29-006db75c47cb`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166409015656', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-454::DEBUG::2012-12-09 12:14:30,636::task::588::TaskManager.Task::(_updateState) Task=`2861543c-dc4d-4ccf-8a29-006db75c47cb`::moving from state preparing -> state finished
>Thread-454::DEBUG::2012-12-09 12:14:30,637::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-454::DEBUG::2012-12-09 12:14:30,637::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-454::DEBUG::2012-12-09 12:14:30,638::task::978::TaskManager.Task::(_decref) Task=`2861543c-dc4d-4ccf-8a29-006db75c47cb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:31,350::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:31,391::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172626 s, 59.3 MB/s\n'; <rc> = 0
>Thread-457::DEBUG::2012-12-09 12:14:33,072::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-457::DEBUG::2012-12-09 12:14:33,073::task::588::TaskManager.Task::(_updateState) Task=`e5df6d1e-a721-4079-bb3c-1cc22f087a47`::moving from state init -> state preparing
>Thread-457::INFO::2012-12-09 12:14:33,074::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-457::INFO::2012-12-09 12:14:33,075::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-457::DEBUG::2012-12-09 12:14:33,075::task::1172::TaskManager.Task::(prepare) Task=`e5df6d1e-a721-4079-bb3c-1cc22f087a47`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-457::DEBUG::2012-12-09 12:14:33,075::task::588::TaskManager.Task::(_updateState) Task=`e5df6d1e-a721-4079-bb3c-1cc22f087a47`::moving from state preparing -> state finished
>Thread-457::DEBUG::2012-12-09 12:14:33,076::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-457::DEBUG::2012-12-09 12:14:33,076::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-457::DEBUG::2012-12-09 12:14:33,077::task::978::TaskManager.Task::(_decref) Task=`e5df6d1e-a721-4079-bb3c-1cc22f087a47`::ref 0 aborting False
>Thread-458::DEBUG::2012-12-09 12:14:33,101::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-458::DEBUG::2012-12-09 12:14:33,102::task::588::TaskManager.Task::(_updateState) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::moving from state init -> state preparing
>Thread-458::INFO::2012-12-09 12:14:33,102::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-458::DEBUG::2012-12-09 12:14:33,103::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1e311b69-1527-446f-94cc-40ad6006eaa9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-458::DEBUG::2012-12-09 12:14:33,103::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-458::DEBUG::2012-12-09 12:14:33,104::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-458::DEBUG::2012-12-09 12:14:33,104::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1e311b69-1527-446f-94cc-40ad6006eaa9`::Granted request
>Thread-458::DEBUG::2012-12-09 12:14:33,105::task::817::TaskManager.Task::(resourceAcquired) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-458::DEBUG::2012-12-09 12:14:33,105::task::978::TaskManager.Task::(_decref) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::ref 1 aborting False
>Thread-458::DEBUG::2012-12-09 12:14:33,107::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-458::DEBUG::2012-12-09 12:14:33,108::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-458::DEBUG::2012-12-09 12:14:33,222::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-458::WARNING::2012-12-09 12:14:33,224::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-458::DEBUG::2012-12-09 12:14:33,224::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-458::WARNING::2012-12-09 12:14:33,225::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-458::INFO::2012-12-09 12:14:33,226::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-458::DEBUG::2012-12-09 12:14:33,226::task::1172::TaskManager.Task::(prepare) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-458::DEBUG::2012-12-09 12:14:33,227::task::588::TaskManager.Task::(_updateState) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::moving from state preparing -> state finished
>Thread-458::DEBUG::2012-12-09 12:14:33,227::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-458::DEBUG::2012-12-09 12:14:33,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-458::DEBUG::2012-12-09 12:14:33,228::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-458::DEBUG::2012-12-09 12:14:33,229::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-458::DEBUG::2012-12-09 12:14:33,229::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-458::DEBUG::2012-12-09 12:14:33,230::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-458::DEBUG::2012-12-09 12:14:33,230::task::978::TaskManager.Task::(_decref) Task=`8b1c701f-2812-48f8-959b-37c62619c88b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:33,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:33,444::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228543 s, 44.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:35,451::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:35,497::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214027 s, 47.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:35,612::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:35,624::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000643081 s, 6.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:36,783::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:36,784::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:36,892::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:36,893::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:36,893::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:36,894::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:37,504::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:37,554::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255225 s, 40.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:39,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:39,597::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152222 s, 67.3 MB/s\n'; <rc> = 0
>Thread-462::DEBUG::2012-12-09 12:14:40,980::task::588::TaskManager.Task::(_updateState) Task=`b8e0f1d1-1b30-4813-8ba6-abd313f07195`::moving from state init -> state preparing
>Thread-462::INFO::2012-12-09 12:14:40,980::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-462::INFO::2012-12-09 12:14:40,980::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126128196716', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-462::DEBUG::2012-12-09 12:14:40,981::task::1172::TaskManager.Task::(prepare) Task=`b8e0f1d1-1b30-4813-8ba6-abd313f07195`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126128196716', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-462::DEBUG::2012-12-09 12:14:40,981::task::588::TaskManager.Task::(_updateState) Task=`b8e0f1d1-1b30-4813-8ba6-abd313f07195`::moving from state preparing -> state finished
>Thread-462::DEBUG::2012-12-09 12:14:40,981::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-462::DEBUG::2012-12-09 12:14:40,981::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-462::DEBUG::2012-12-09 12:14:40,981::task::978::TaskManager.Task::(_decref) Task=`b8e0f1d1-1b30-4813-8ba6-abd313f07195`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:41,604::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:41,654::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025783 s, 39.7 MB/s\n'; <rc> = 0
>Thread-465::DEBUG::2012-12-09 12:14:43,366::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-465::DEBUG::2012-12-09 12:14:43,367::task::588::TaskManager.Task::(_updateState) Task=`ce018f22-6dac-4d9f-9c89-92a1d8f66207`::moving from state init -> state preparing
>Thread-465::INFO::2012-12-09 12:14:43,367::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-465::INFO::2012-12-09 12:14:43,368::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-465::DEBUG::2012-12-09 12:14:43,368::task::1172::TaskManager.Task::(prepare) Task=`ce018f22-6dac-4d9f-9c89-92a1d8f66207`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-465::DEBUG::2012-12-09 12:14:43,369::task::588::TaskManager.Task::(_updateState) Task=`ce018f22-6dac-4d9f-9c89-92a1d8f66207`::moving from state preparing -> state finished
>Thread-465::DEBUG::2012-12-09 12:14:43,369::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-465::DEBUG::2012-12-09 12:14:43,369::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-465::DEBUG::2012-12-09 12:14:43,370::task::978::TaskManager.Task::(_decref) Task=`ce018f22-6dac-4d9f-9c89-92a1d8f66207`::ref 0 aborting False
>Thread-466::DEBUG::2012-12-09 12:14:43,452::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-466::DEBUG::2012-12-09 12:14:43,452::task::588::TaskManager.Task::(_updateState) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::moving from state init -> state preparing
>Thread-466::INFO::2012-12-09 12:14:43,452::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-466::DEBUG::2012-12-09 12:14:43,453::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`857c5078-e5ac-4983-9ca2-1aefdcc1fc9f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-466::DEBUG::2012-12-09 12:14:43,453::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-466::DEBUG::2012-12-09 12:14:43,454::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-466::DEBUG::2012-12-09 12:14:43,454::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`857c5078-e5ac-4983-9ca2-1aefdcc1fc9f`::Granted request
>Thread-466::DEBUG::2012-12-09 12:14:43,454::task::817::TaskManager.Task::(resourceAcquired) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-466::DEBUG::2012-12-09 12:14:43,454::task::978::TaskManager.Task::(_decref) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::ref 1 aborting False
>Thread-466::DEBUG::2012-12-09 12:14:43,455::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-466::DEBUG::2012-12-09 12:14:43,456::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-466::DEBUG::2012-12-09 12:14:43,559::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-466::WARNING::2012-12-09 12:14:43,560::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-466::DEBUG::2012-12-09 12:14:43,560::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-466::WARNING::2012-12-09 12:14:43,561::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-466::INFO::2012-12-09 12:14:43,561::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-466::DEBUG::2012-12-09 12:14:43,561::task::1172::TaskManager.Task::(prepare) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-466::DEBUG::2012-12-09 12:14:43,562::task::588::TaskManager.Task::(_updateState) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::moving from state preparing -> state finished
>Thread-466::DEBUG::2012-12-09 12:14:43,562::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-466::DEBUG::2012-12-09 12:14:43,562::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-466::DEBUG::2012-12-09 12:14:43,562::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-466::DEBUG::2012-12-09 12:14:43,562::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-466::DEBUG::2012-12-09 12:14:43,563::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-466::DEBUG::2012-12-09 12:14:43,563::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-466::DEBUG::2012-12-09 12:14:43,563::task::978::TaskManager.Task::(_decref) Task=`5132733e-0eda-4c1a-b6b0-5f06db3f0750`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:43,661::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:43,706::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215915 s, 47.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:45,627::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:45,637::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000703262 s, 5.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:45,713::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:45,750::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161659 s, 63.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:46,896::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:46,897::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:47,011::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:47,012::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:47,012::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:47,013::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:47,756::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:47,799::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177955 s, 57.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:49,806::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:49,848::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183416 s, 55.8 MB/s\n'; <rc> = 0
>Thread-470::DEBUG::2012-12-09 12:14:51,489::task::588::TaskManager.Task::(_updateState) Task=`22bd2e02-7c91-4a9f-b430-5b213117c475`::moving from state init -> state preparing
>Thread-470::INFO::2012-12-09 12:14:51,490::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-470::INFO::2012-12-09 12:14:51,491::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107080936432', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-470::DEBUG::2012-12-09 12:14:51,491::task::1172::TaskManager.Task::(prepare) Task=`22bd2e02-7c91-4a9f-b430-5b213117c475`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107080936432', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-470::DEBUG::2012-12-09 12:14:51,492::task::588::TaskManager.Task::(_updateState) Task=`22bd2e02-7c91-4a9f-b430-5b213117c475`::moving from state preparing -> state finished
>Thread-470::DEBUG::2012-12-09 12:14:51,492::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-470::DEBUG::2012-12-09 12:14:51,492::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-470::DEBUG::2012-12-09 12:14:51,493::task::978::TaskManager.Task::(_decref) Task=`22bd2e02-7c91-4a9f-b430-5b213117c475`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:51,855::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:51,908::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282731 s, 36.2 MB/s\n'; <rc> = 0
>Thread-472::DEBUG::2012-12-09 12:14:53,712::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-472::DEBUG::2012-12-09 12:14:53,713::task::588::TaskManager.Task::(_updateState) Task=`9cd87f4a-594f-4bd5-b764-8fe75504ae9f`::moving from state init -> state preparing
>Thread-472::INFO::2012-12-09 12:14:53,714::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-472::INFO::2012-12-09 12:14:53,715::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-472::DEBUG::2012-12-09 12:14:53,715::task::1172::TaskManager.Task::(prepare) Task=`9cd87f4a-594f-4bd5-b764-8fe75504ae9f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-472::DEBUG::2012-12-09 12:14:53,715::task::588::TaskManager.Task::(_updateState) Task=`9cd87f4a-594f-4bd5-b764-8fe75504ae9f`::moving from state preparing -> state finished
>Thread-472::DEBUG::2012-12-09 12:14:53,716::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-472::DEBUG::2012-12-09 12:14:53,716::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-472::DEBUG::2012-12-09 12:14:53,717::task::978::TaskManager.Task::(_decref) Task=`9cd87f4a-594f-4bd5-b764-8fe75504ae9f`::ref 0 aborting False
>Thread-473::DEBUG::2012-12-09 12:14:53,741::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-473::DEBUG::2012-12-09 12:14:53,741::task::588::TaskManager.Task::(_updateState) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::moving from state init -> state preparing
>Thread-473::INFO::2012-12-09 12:14:53,742::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-473::DEBUG::2012-12-09 12:14:53,742::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25c4498f-3752-4437-8d6a-dbcf30d5b709`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-473::DEBUG::2012-12-09 12:14:53,742::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-473::DEBUG::2012-12-09 12:14:53,743::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-473::DEBUG::2012-12-09 12:14:53,743::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25c4498f-3752-4437-8d6a-dbcf30d5b709`::Granted request
>Thread-473::DEBUG::2012-12-09 12:14:53,743::task::817::TaskManager.Task::(resourceAcquired) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-473::DEBUG::2012-12-09 12:14:53,744::task::978::TaskManager.Task::(_decref) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::ref 1 aborting False
>Thread-473::DEBUG::2012-12-09 12:14:53,745::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-473::DEBUG::2012-12-09 12:14:53,746::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-473::DEBUG::2012-12-09 12:14:53,846::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-473::WARNING::2012-12-09 12:14:53,847::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-473::DEBUG::2012-12-09 12:14:53,848::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-473::WARNING::2012-12-09 12:14:53,849::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-473::INFO::2012-12-09 12:14:53,849::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-473::DEBUG::2012-12-09 12:14:53,850::task::1172::TaskManager.Task::(prepare) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-473::DEBUG::2012-12-09 12:14:53,850::task::588::TaskManager.Task::(_updateState) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::moving from state preparing -> state finished
>Thread-473::DEBUG::2012-12-09 12:14:53,851::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-473::DEBUG::2012-12-09 12:14:53,851::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-473::DEBUG::2012-12-09 12:14:53,852::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-473::DEBUG::2012-12-09 12:14:53,852::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-473::DEBUG::2012-12-09 12:14:53,852::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-473::DEBUG::2012-12-09 12:14:53,853::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-473::DEBUG::2012-12-09 12:14:53,853::task::978::TaskManager.Task::(_decref) Task=`7ccf7c8c-41dd-44ab-8c06-2f8d1ccf257a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:14:53,915::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:53,965::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0253512 s, 40.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:14:55,640::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:14:55,653::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00103649 s, 4.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:14:55,972::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:56,025::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268223 s, 38.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:14:57,014::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:14:57,015::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:14:57,131::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:14:57,133::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:14:57,133::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:14:57,134::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:14:58,032::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:14:58,084::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266112 s, 38.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:00,090::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:00,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276516 s, 37.0 MB/s\n'; <rc> = 0
>Thread-478::DEBUG::2012-12-09 12:15:01,899::task::588::TaskManager.Task::(_updateState) Task=`56903247-67f0-4839-83f9-75b5a790be4d`::moving from state init -> state preparing
>Thread-478::INFO::2012-12-09 12:15:01,899::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-478::INFO::2012-12-09 12:15:01,900::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129249095917', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.8', 'code': 358, 'valid': False}}
>Thread-478::DEBUG::2012-12-09 12:15:01,900::task::1172::TaskManager.Task::(prepare) Task=`56903247-67f0-4839-83f9-75b5a790be4d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129249095917', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.8', 'code': 358, 'valid': False}}
>Thread-478::DEBUG::2012-12-09 12:15:01,901::task::588::TaskManager.Task::(_updateState) Task=`56903247-67f0-4839-83f9-75b5a790be4d`::moving from state preparing -> state finished
>Thread-478::DEBUG::2012-12-09 12:15:01,901::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-478::DEBUG::2012-12-09 12:15:01,902::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-478::DEBUG::2012-12-09 12:15:01,902::task::978::TaskManager.Task::(_decref) Task=`56903247-67f0-4839-83f9-75b5a790be4d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:02,149::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:02,203::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0288699 s, 35.5 MB/s\n'; <rc> = 0
>Thread-480::DEBUG::2012-12-09 12:15:04,013::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-480::DEBUG::2012-12-09 12:15:04,014::task::588::TaskManager.Task::(_updateState) Task=`a2ca0b52-2ba8-4450-8f5d-50869cf05883`::moving from state init -> state preparing
>Thread-480::INFO::2012-12-09 12:15:04,015::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-480::INFO::2012-12-09 12:15:04,015::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-480::DEBUG::2012-12-09 12:15:04,016::task::1172::TaskManager.Task::(prepare) Task=`a2ca0b52-2ba8-4450-8f5d-50869cf05883`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-480::DEBUG::2012-12-09 12:15:04,016::task::588::TaskManager.Task::(_updateState) Task=`a2ca0b52-2ba8-4450-8f5d-50869cf05883`::moving from state preparing -> state finished
>Thread-480::DEBUG::2012-12-09 12:15:04,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-480::DEBUG::2012-12-09 12:15:04,017::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-480::DEBUG::2012-12-09 12:15:04,017::task::978::TaskManager.Task::(_decref) Task=`a2ca0b52-2ba8-4450-8f5d-50869cf05883`::ref 0 aborting False
>Thread-481::DEBUG::2012-12-09 12:15:04,033::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-481::DEBUG::2012-12-09 12:15:04,034::task::588::TaskManager.Task::(_updateState) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::moving from state init -> state preparing
>Thread-481::INFO::2012-12-09 12:15:04,034::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-481::DEBUG::2012-12-09 12:15:04,034::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a31d0290-9a1b-405d-b150-0f6d7bbd4c2a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-481::DEBUG::2012-12-09 12:15:04,035::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-481::DEBUG::2012-12-09 12:15:04,035::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-481::DEBUG::2012-12-09 12:15:04,035::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a31d0290-9a1b-405d-b150-0f6d7bbd4c2a`::Granted request
>Thread-481::DEBUG::2012-12-09 12:15:04,036::task::817::TaskManager.Task::(resourceAcquired) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-481::DEBUG::2012-12-09 12:15:04,036::task::978::TaskManager.Task::(_decref) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::ref 1 aborting False
>Thread-481::DEBUG::2012-12-09 12:15:04,037::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-481::DEBUG::2012-12-09 12:15:04,037::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-481::DEBUG::2012-12-09 12:15:04,150::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-481::WARNING::2012-12-09 12:15:04,152::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-481::DEBUG::2012-12-09 12:15:04,152::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-481::WARNING::2012-12-09 12:15:04,153::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-481::INFO::2012-12-09 12:15:04,153::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-481::DEBUG::2012-12-09 12:15:04,153::task::1172::TaskManager.Task::(prepare) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-481::DEBUG::2012-12-09 12:15:04,154::task::588::TaskManager.Task::(_updateState) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::moving from state preparing -> state finished
>Thread-481::DEBUG::2012-12-09 12:15:04,154::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-481::DEBUG::2012-12-09 12:15:04,154::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-481::DEBUG::2012-12-09 12:15:04,154::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-481::DEBUG::2012-12-09 12:15:04,155::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-481::DEBUG::2012-12-09 12:15:04,155::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-481::DEBUG::2012-12-09 12:15:04,155::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-481::DEBUG::2012-12-09 12:15:04,155::task::978::TaskManager.Task::(_decref) Task=`457e83b4-5f07-4d3f-8aba-d8946ee44d43`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:04,209::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:04,256::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222349 s, 46.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:05,656::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:05,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000902773 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:06,263::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:06,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271483 s, 37.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:07,136::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:07,137::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:07,251::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:07,252::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:07,252::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:07,253::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:08,321::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:08,377::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275014 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:10,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:10,435::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254568 s, 40.2 MB/s\n'; <rc> = 0
>Thread-486::DEBUG::2012-12-09 12:15:12,212::task::588::TaskManager.Task::(_updateState) Task=`82fe5014-5c7d-4f07-a875-1a0bdb245b06`::moving from state init -> state preparing
>Thread-486::INFO::2012-12-09 12:15:12,212::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-486::INFO::2012-12-09 12:15:12,213::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152831077576', 'lastCheck': '6.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-486::DEBUG::2012-12-09 12:15:12,213::task::1172::TaskManager.Task::(prepare) Task=`82fe5014-5c7d-4f07-a875-1a0bdb245b06`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152831077576', 'lastCheck': '6.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-486::DEBUG::2012-12-09 12:15:12,214::task::588::TaskManager.Task::(_updateState) Task=`82fe5014-5c7d-4f07-a875-1a0bdb245b06`::moving from state preparing -> state finished
>Thread-486::DEBUG::2012-12-09 12:15:12,214::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-486::DEBUG::2012-12-09 12:15:12,215::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-486::DEBUG::2012-12-09 12:15:12,215::task::978::TaskManager.Task::(_decref) Task=`82fe5014-5c7d-4f07-a875-1a0bdb245b06`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:12,442::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:12,486::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199016 s, 51.5 MB/s\n'; <rc> = 0
>Thread-488::DEBUG::2012-12-09 12:15:14,232::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-488::DEBUG::2012-12-09 12:15:14,233::task::588::TaskManager.Task::(_updateState) Task=`657d3ce6-6146-447f-9553-f8a7c0673ae5`::moving from state init -> state preparing
>Thread-488::INFO::2012-12-09 12:15:14,234::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-488::INFO::2012-12-09 12:15:14,234::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-488::DEBUG::2012-12-09 12:15:14,235::task::1172::TaskManager.Task::(prepare) Task=`657d3ce6-6146-447f-9553-f8a7c0673ae5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-488::DEBUG::2012-12-09 12:15:14,235::task::588::TaskManager.Task::(_updateState) Task=`657d3ce6-6146-447f-9553-f8a7c0673ae5`::moving from state preparing -> state finished
>Thread-488::DEBUG::2012-12-09 12:15:14,236::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-488::DEBUG::2012-12-09 12:15:14,236::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-488::DEBUG::2012-12-09 12:15:14,236::task::978::TaskManager.Task::(_decref) Task=`657d3ce6-6146-447f-9553-f8a7c0673ae5`::ref 0 aborting False
>Thread-489::DEBUG::2012-12-09 12:15:14,260::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-489::DEBUG::2012-12-09 12:15:14,261::task::588::TaskManager.Task::(_updateState) Task=`8124964a-88cb-4990-9589-405c4a05991d`::moving from state init -> state preparing
>Thread-489::INFO::2012-12-09 12:15:14,261::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-489::DEBUG::2012-12-09 12:15:14,262::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bc5ad370-824f-4fbf-a228-62e9af7f1294`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-489::DEBUG::2012-12-09 12:15:14,263::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-489::DEBUG::2012-12-09 12:15:14,263::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-489::DEBUG::2012-12-09 12:15:14,263::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bc5ad370-824f-4fbf-a228-62e9af7f1294`::Granted request
>Thread-489::DEBUG::2012-12-09 12:15:14,264::task::817::TaskManager.Task::(resourceAcquired) Task=`8124964a-88cb-4990-9589-405c4a05991d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-489::DEBUG::2012-12-09 12:15:14,265::task::978::TaskManager.Task::(_decref) Task=`8124964a-88cb-4990-9589-405c4a05991d`::ref 1 aborting False
>Thread-489::DEBUG::2012-12-09 12:15:14,266::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-489::DEBUG::2012-12-09 12:15:14,267::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-489::DEBUG::2012-12-09 12:15:14,381::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-489::WARNING::2012-12-09 12:15:14,382::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-489::DEBUG::2012-12-09 12:15:14,382::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-489::WARNING::2012-12-09 12:15:14,383::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-489::INFO::2012-12-09 12:15:14,384::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-489::DEBUG::2012-12-09 12:15:14,384::task::1172::TaskManager.Task::(prepare) Task=`8124964a-88cb-4990-9589-405c4a05991d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-489::DEBUG::2012-12-09 12:15:14,385::task::588::TaskManager.Task::(_updateState) Task=`8124964a-88cb-4990-9589-405c4a05991d`::moving from state preparing -> state finished
>Thread-489::DEBUG::2012-12-09 12:15:14,385::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-489::DEBUG::2012-12-09 12:15:14,386::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-489::DEBUG::2012-12-09 12:15:14,386::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-489::DEBUG::2012-12-09 12:15:14,387::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-489::DEBUG::2012-12-09 12:15:14,387::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-489::DEBUG::2012-12-09 12:15:14,387::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-489::DEBUG::2012-12-09 12:15:14,388::task::978::TaskManager.Task::(_decref) Task=`8124964a-88cb-4990-9589-405c4a05991d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:14,493::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:14,537::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209372 s, 48.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:15,674::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:15,689::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000921666 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:16,543::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:16,595::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0253512 s, 40.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:17,254::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:17,255::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:17,375::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:17,377::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:17,377::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:17,379::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:18,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:18,641::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161695 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:20,648::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:20,694::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178609 s, 57.3 MB/s\n'; <rc> = 0
>Thread-494::DEBUG::2012-12-09 12:15:22,614::task::588::TaskManager.Task::(_updateState) Task=`829d409e-cd46-44d6-9e04-1fefb0f3b857`::moving from state init -> state preparing
>Thread-494::INFO::2012-12-09 12:15:22,615::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-494::INFO::2012-12-09 12:15:22,616::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162971019745', 'lastCheck': '6.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-494::DEBUG::2012-12-09 12:15:22,616::task::1172::TaskManager.Task::(prepare) Task=`829d409e-cd46-44d6-9e04-1fefb0f3b857`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162971019745', 'lastCheck': '6.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-494::DEBUG::2012-12-09 12:15:22,617::task::588::TaskManager.Task::(_updateState) Task=`829d409e-cd46-44d6-9e04-1fefb0f3b857`::moving from state preparing -> state finished
>Thread-494::DEBUG::2012-12-09 12:15:22,617::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-494::DEBUG::2012-12-09 12:15:22,617::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-494::DEBUG::2012-12-09 12:15:22,618::task::978::TaskManager.Task::(_decref) Task=`829d409e-cd46-44d6-9e04-1fefb0f3b857`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:22,700::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:22,739::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154179 s, 66.4 MB/s\n'; <rc> = 0
>Thread-496::DEBUG::2012-12-09 12:15:24,534::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-496::DEBUG::2012-12-09 12:15:24,535::task::588::TaskManager.Task::(_updateState) Task=`707b0664-8539-4f03-9f0c-91a25f47d51a`::moving from state init -> state preparing
>Thread-496::INFO::2012-12-09 12:15:24,535::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-496::INFO::2012-12-09 12:15:24,536::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-496::DEBUG::2012-12-09 12:15:24,536::task::1172::TaskManager.Task::(prepare) Task=`707b0664-8539-4f03-9f0c-91a25f47d51a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-496::DEBUG::2012-12-09 12:15:24,536::task::588::TaskManager.Task::(_updateState) Task=`707b0664-8539-4f03-9f0c-91a25f47d51a`::moving from state preparing -> state finished
>Thread-496::DEBUG::2012-12-09 12:15:24,537::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-496::DEBUG::2012-12-09 12:15:24,537::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-496::DEBUG::2012-12-09 12:15:24,538::task::978::TaskManager.Task::(_decref) Task=`707b0664-8539-4f03-9f0c-91a25f47d51a`::ref 0 aborting False
>Thread-497::DEBUG::2012-12-09 12:15:24,562::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-497::DEBUG::2012-12-09 12:15:24,563::task::588::TaskManager.Task::(_updateState) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::moving from state init -> state preparing
>Thread-497::INFO::2012-12-09 12:15:24,564::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-497::DEBUG::2012-12-09 12:15:24,564::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`91aaa1f0-7dc1-48ed-92f7-2400d8fbbe06`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-497::DEBUG::2012-12-09 12:15:24,564::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-497::DEBUG::2012-12-09 12:15:24,565::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-497::DEBUG::2012-12-09 12:15:24,565::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`91aaa1f0-7dc1-48ed-92f7-2400d8fbbe06`::Granted request
>Thread-497::DEBUG::2012-12-09 12:15:24,566::task::817::TaskManager.Task::(resourceAcquired) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-497::DEBUG::2012-12-09 12:15:24,566::task::978::TaskManager.Task::(_decref) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::ref 1 aborting False
>Thread-497::DEBUG::2012-12-09 12:15:24,568::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-497::DEBUG::2012-12-09 12:15:24,568::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-497::DEBUG::2012-12-09 12:15:24,681::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-497::WARNING::2012-12-09 12:15:24,683::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-497::DEBUG::2012-12-09 12:15:24,683::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-497::WARNING::2012-12-09 12:15:24,683::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-497::INFO::2012-12-09 12:15:24,684::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-497::DEBUG::2012-12-09 12:15:24,684::task::1172::TaskManager.Task::(prepare) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-497::DEBUG::2012-12-09 12:15:24,684::task::588::TaskManager.Task::(_updateState) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::moving from state preparing -> state finished
>Thread-497::DEBUG::2012-12-09 12:15:24,684::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-497::DEBUG::2012-12-09 12:15:24,684::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-497::DEBUG::2012-12-09 12:15:24,685::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-497::DEBUG::2012-12-09 12:15:24,685::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-497::DEBUG::2012-12-09 12:15:24,685::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-497::DEBUG::2012-12-09 12:15:24,685::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-497::DEBUG::2012-12-09 12:15:24,686::task::978::TaskManager.Task::(_decref) Task=`6add0ae9-f598-4a9d-b5e5-77a83ffcb18a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:24,745::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:24,795::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247463 s, 41.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:25,693::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:25,708::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000848774 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:26,802::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:26,850::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193198 s, 53.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:27,380::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:27,381::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:27,497::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:27,499::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:27,500::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:27,500::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:28,856::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:28,904::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233057 s, 43.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:30,910::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:30,959::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208891 s, 49.0 MB/s\n'; <rc> = 0
>Thread-502::DEBUG::2012-12-09 12:15:32,955::task::588::TaskManager.Task::(_updateState) Task=`23ed2c0a-e5e7-4ef0-a9e6-5d2862cd7819`::moving from state init -> state preparing
>Thread-502::INFO::2012-12-09 12:15:32,955::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-502::INFO::2012-12-09 12:15:32,955::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0165858268738', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-502::DEBUG::2012-12-09 12:15:32,955::task::1172::TaskManager.Task::(prepare) Task=`23ed2c0a-e5e7-4ef0-a9e6-5d2862cd7819`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0165858268738', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-502::DEBUG::2012-12-09 12:15:32,956::task::588::TaskManager.Task::(_updateState) Task=`23ed2c0a-e5e7-4ef0-a9e6-5d2862cd7819`::moving from state preparing -> state finished
>Thread-502::DEBUG::2012-12-09 12:15:32,956::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-502::DEBUG::2012-12-09 12:15:32,956::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-502::DEBUG::2012-12-09 12:15:32,956::task::978::TaskManager.Task::(_decref) Task=`23ed2c0a-e5e7-4ef0-a9e6-5d2862cd7819`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:32,966::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:33,023::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282926 s, 36.2 MB/s\n'; <rc> = 0
>Thread-504::DEBUG::2012-12-09 12:15:34,833::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-504::DEBUG::2012-12-09 12:15:34,834::task::588::TaskManager.Task::(_updateState) Task=`a0d6aabe-61ca-4f66-8de0-2498a2cdac09`::moving from state init -> state preparing
>Thread-504::INFO::2012-12-09 12:15:34,835::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-504::INFO::2012-12-09 12:15:34,835::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-504::DEBUG::2012-12-09 12:15:34,836::task::1172::TaskManager.Task::(prepare) Task=`a0d6aabe-61ca-4f66-8de0-2498a2cdac09`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-504::DEBUG::2012-12-09 12:15:34,836::task::588::TaskManager.Task::(_updateState) Task=`a0d6aabe-61ca-4f66-8de0-2498a2cdac09`::moving from state preparing -> state finished
>Thread-504::DEBUG::2012-12-09 12:15:34,837::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-504::DEBUG::2012-12-09 12:15:34,837::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-504::DEBUG::2012-12-09 12:15:34,838::task::978::TaskManager.Task::(_decref) Task=`a0d6aabe-61ca-4f66-8de0-2498a2cdac09`::ref 0 aborting False
>Thread-505::DEBUG::2012-12-09 12:15:34,853::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-505::DEBUG::2012-12-09 12:15:34,853::task::588::TaskManager.Task::(_updateState) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::moving from state init -> state preparing
>Thread-505::INFO::2012-12-09 12:15:34,854::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-505::DEBUG::2012-12-09 12:15:34,855::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5683467e-4dfb-4e8f-842c-24a454a029ff`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-505::DEBUG::2012-12-09 12:15:34,855::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-505::DEBUG::2012-12-09 12:15:34,856::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-505::DEBUG::2012-12-09 12:15:34,856::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5683467e-4dfb-4e8f-842c-24a454a029ff`::Granted request
>Thread-505::DEBUG::2012-12-09 12:15:34,857::task::817::TaskManager.Task::(resourceAcquired) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-505::DEBUG::2012-12-09 12:15:34,857::task::978::TaskManager.Task::(_decref) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::ref 1 aborting False
>Thread-505::DEBUG::2012-12-09 12:15:34,859::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-505::DEBUG::2012-12-09 12:15:34,860::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-505::DEBUG::2012-12-09 12:15:34,968::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-505::WARNING::2012-12-09 12:15:34,970::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-505::DEBUG::2012-12-09 12:15:34,970::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-505::WARNING::2012-12-09 12:15:34,971::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-505::INFO::2012-12-09 12:15:34,971::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-505::DEBUG::2012-12-09 12:15:34,971::task::1172::TaskManager.Task::(prepare) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-505::DEBUG::2012-12-09 12:15:34,971::task::588::TaskManager.Task::(_updateState) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::moving from state preparing -> state finished
>Thread-505::DEBUG::2012-12-09 12:15:34,972::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-505::DEBUG::2012-12-09 12:15:34,972::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-505::DEBUG::2012-12-09 12:15:34,972::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-505::DEBUG::2012-12-09 12:15:34,972::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-505::DEBUG::2012-12-09 12:15:34,972::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-505::DEBUG::2012-12-09 12:15:34,973::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-505::DEBUG::2012-12-09 12:15:34,973::task::978::TaskManager.Task::(_decref) Task=`7e0a33ad-4938-4ad6-b7e0-0c921bb7d182`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:35,029::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:35,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220292 s, 46.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:35,712::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:35,728::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000744408 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:37,083::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:37,129::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203985 s, 50.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:37,501::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:37,503::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:37,620::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:37,621::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:37,622::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:37,622::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:39,135::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:39,184::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184104 s, 55.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:41,192::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:41,247::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241638 s, 42.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:43,254::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-510::DEBUG::2012-12-09 12:15:43,275::task::588::TaskManager.Task::(_updateState) Task=`d231347f-9210-4bfb-9086-5a46b7601d66`::moving from state init -> state preparing
>Thread-510::INFO::2012-12-09 12:15:43,278::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-510::INFO::2012-12-09 12:15:43,280::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167920589447', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-510::DEBUG::2012-12-09 12:15:43,282::task::1172::TaskManager.Task::(prepare) Task=`d231347f-9210-4bfb-9086-5a46b7601d66`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167920589447', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-510::DEBUG::2012-12-09 12:15:43,283::task::588::TaskManager.Task::(_updateState) Task=`d231347f-9210-4bfb-9086-5a46b7601d66`::moving from state preparing -> state finished
>Thread-510::DEBUG::2012-12-09 12:15:43,285::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-510::DEBUG::2012-12-09 12:15:43,287::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-510::DEBUG::2012-12-09 12:15:43,288::task::978::TaskManager.Task::(_decref) Task=`d231347f-9210-4bfb-9086-5a46b7601d66`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:43,337::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0550303 s, 18.6 MB/s\n'; <rc> = 0
>Thread-512::DEBUG::2012-12-09 12:15:45,188::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-512::DEBUG::2012-12-09 12:15:45,189::task::588::TaskManager.Task::(_updateState) Task=`958fea36-b1c2-422c-baef-4d2141cd92b9`::moving from state init -> state preparing
>Thread-512::INFO::2012-12-09 12:15:45,189::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-512::INFO::2012-12-09 12:15:45,190::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-512::DEBUG::2012-12-09 12:15:45,191::task::1172::TaskManager.Task::(prepare) Task=`958fea36-b1c2-422c-baef-4d2141cd92b9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-512::DEBUG::2012-12-09 12:15:45,191::task::588::TaskManager.Task::(_updateState) Task=`958fea36-b1c2-422c-baef-4d2141cd92b9`::moving from state preparing -> state finished
>Thread-512::DEBUG::2012-12-09 12:15:45,191::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-512::DEBUG::2012-12-09 12:15:45,192::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-512::DEBUG::2012-12-09 12:15:45,192::task::978::TaskManager.Task::(_decref) Task=`958fea36-b1c2-422c-baef-4d2141cd92b9`::ref 0 aborting False
>Thread-513::DEBUG::2012-12-09 12:15:45,214::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-513::DEBUG::2012-12-09 12:15:45,215::task::588::TaskManager.Task::(_updateState) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::moving from state init -> state preparing
>Thread-513::INFO::2012-12-09 12:15:45,215::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-513::DEBUG::2012-12-09 12:15:45,216::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7ba2d5ea-bf5c-4905-9919-bb2e940e996e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-513::DEBUG::2012-12-09 12:15:45,216::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-513::DEBUG::2012-12-09 12:15:45,217::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-513::DEBUG::2012-12-09 12:15:45,217::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7ba2d5ea-bf5c-4905-9919-bb2e940e996e`::Granted request
>Thread-513::DEBUG::2012-12-09 12:15:45,218::task::817::TaskManager.Task::(resourceAcquired) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-513::DEBUG::2012-12-09 12:15:45,218::task::978::TaskManager.Task::(_decref) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::ref 1 aborting False
>Thread-513::DEBUG::2012-12-09 12:15:45,220::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-513::DEBUG::2012-12-09 12:15:45,220::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-513::DEBUG::2012-12-09 12:15:45,341::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-513::WARNING::2012-12-09 12:15:45,342::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-513::DEBUG::2012-12-09 12:15:45,342::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Dummy-321::DEBUG::2012-12-09 12:15:45,343::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-513::WARNING::2012-12-09 12:15:45,348::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-513::INFO::2012-12-09 12:15:45,348::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-513::DEBUG::2012-12-09 12:15:45,349::task::1172::TaskManager.Task::(prepare) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-513::DEBUG::2012-12-09 12:15:45,349::task::588::TaskManager.Task::(_updateState) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::moving from state preparing -> state finished
>Thread-513::DEBUG::2012-12-09 12:15:45,349::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-513::DEBUG::2012-12-09 12:15:45,349::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-513::DEBUG::2012-12-09 12:15:45,349::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-513::DEBUG::2012-12-09 12:15:45,350::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-513::DEBUG::2012-12-09 12:15:45,350::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-513::DEBUG::2012-12-09 12:15:45,350::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-513::DEBUG::2012-12-09 12:15:45,350::task::978::TaskManager.Task::(_decref) Task=`b3088dab-52d5-4738-8b74-33818c36944f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:45,389::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195372 s, 52.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:45,732::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:45,747::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000922627 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:47,396::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:47,442::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195571 s, 52.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:47,623::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:47,624::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:47,740::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:47,741::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:47,741::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:47,742::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:49,449::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:49,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230395 s, 44.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:51,504::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:51,554::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02664 s, 38.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:53,559::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:53,613::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244975 s, 41.8 MB/s\n'; <rc> = 0
>Thread-518::DEBUG::2012-12-09 12:15:53,629::task::588::TaskManager.Task::(_updateState) Task=`458729cc-aa47-4734-85d5-6ab08f436c0d`::moving from state init -> state preparing
>Thread-518::INFO::2012-12-09 12:15:53,630::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-518::INFO::2012-12-09 12:15:53,630::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160851478577', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-518::DEBUG::2012-12-09 12:15:53,631::task::1172::TaskManager.Task::(prepare) Task=`458729cc-aa47-4734-85d5-6ab08f436c0d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160851478577', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-518::DEBUG::2012-12-09 12:15:53,631::task::588::TaskManager.Task::(_updateState) Task=`458729cc-aa47-4734-85d5-6ab08f436c0d`::moving from state preparing -> state finished
>Thread-518::DEBUG::2012-12-09 12:15:53,631::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-518::DEBUG::2012-12-09 12:15:53,632::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-518::DEBUG::2012-12-09 12:15:53,632::task::978::TaskManager.Task::(_decref) Task=`458729cc-aa47-4734-85d5-6ab08f436c0d`::ref 0 aborting False
>Thread-520::DEBUG::2012-12-09 12:15:55,411::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-520::DEBUG::2012-12-09 12:15:55,412::task::588::TaskManager.Task::(_updateState) Task=`49611d89-7a2d-485c-91a2-8231a52fc598`::moving from state init -> state preparing
>Thread-520::INFO::2012-12-09 12:15:55,412::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-520::INFO::2012-12-09 12:15:55,412::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-520::DEBUG::2012-12-09 12:15:55,413::task::1172::TaskManager.Task::(prepare) Task=`49611d89-7a2d-485c-91a2-8231a52fc598`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-520::DEBUG::2012-12-09 12:15:55,413::task::588::TaskManager.Task::(_updateState) Task=`49611d89-7a2d-485c-91a2-8231a52fc598`::moving from state preparing -> state finished
>Thread-520::DEBUG::2012-12-09 12:15:55,413::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-520::DEBUG::2012-12-09 12:15:55,413::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-520::DEBUG::2012-12-09 12:15:55,413::task::978::TaskManager.Task::(_decref) Task=`49611d89-7a2d-485c-91a2-8231a52fc598`::ref 0 aborting False
>Thread-521::DEBUG::2012-12-09 12:15:55,428::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-521::DEBUG::2012-12-09 12:15:55,429::task::588::TaskManager.Task::(_updateState) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::moving from state init -> state preparing
>Thread-521::INFO::2012-12-09 12:15:55,429::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-521::DEBUG::2012-12-09 12:15:55,429::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2953c602-7c31-443d-a807-79336d66ff69`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-521::DEBUG::2012-12-09 12:15:55,430::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-521::DEBUG::2012-12-09 12:15:55,430::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-521::DEBUG::2012-12-09 12:15:55,430::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2953c602-7c31-443d-a807-79336d66ff69`::Granted request
>Thread-521::DEBUG::2012-12-09 12:15:55,430::task::817::TaskManager.Task::(resourceAcquired) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-521::DEBUG::2012-12-09 12:15:55,431::task::978::TaskManager.Task::(_decref) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::ref 1 aborting False
>Thread-521::DEBUG::2012-12-09 12:15:55,431::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-521::DEBUG::2012-12-09 12:15:55,432::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-521::DEBUG::2012-12-09 12:15:55,541::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-521::WARNING::2012-12-09 12:15:55,542::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-521::DEBUG::2012-12-09 12:15:55,542::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-521::WARNING::2012-12-09 12:15:55,542::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-521::INFO::2012-12-09 12:15:55,543::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-521::DEBUG::2012-12-09 12:15:55,543::task::1172::TaskManager.Task::(prepare) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-521::DEBUG::2012-12-09 12:15:55,543::task::588::TaskManager.Task::(_updateState) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::moving from state preparing -> state finished
>Thread-521::DEBUG::2012-12-09 12:15:55,543::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-521::DEBUG::2012-12-09 12:15:55,544::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-521::DEBUG::2012-12-09 12:15:55,544::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-521::DEBUG::2012-12-09 12:15:55,544::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-521::DEBUG::2012-12-09 12:15:55,544::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-521::DEBUG::2012-12-09 12:15:55,544::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-521::DEBUG::2012-12-09 12:15:55,545::task::978::TaskManager.Task::(_decref) Task=`faa0ff1c-867e-4df5-ba70-fb6868ecf8ba`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:15:55,620::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:55,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280867 s, 36.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:15:55,753::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:15:55,766::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00090705 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:15:57,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:57,730::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169845 s, 60.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:15:57,743::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:15:57,743::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:15:57,866::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:15:57,867::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:15:57,867::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:15:57,868::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:15:59,738::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:15:59,785::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240752 s, 42.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:01,792::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:01,844::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273785 s, 37.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:03,851::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:03,897::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022264 s, 46.0 MB/s\n'; <rc> = 0
>Thread-526::DEBUG::2012-12-09 12:16:03,980::task::588::TaskManager.Task::(_updateState) Task=`84b03292-fd01-4ff9-89dd-c72b5245e1cf`::moving from state init -> state preparing
>Thread-526::INFO::2012-12-09 12:16:03,980::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-526::INFO::2012-12-09 12:16:03,980::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0142090320587', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-526::DEBUG::2012-12-09 12:16:03,981::task::1172::TaskManager.Task::(prepare) Task=`84b03292-fd01-4ff9-89dd-c72b5245e1cf`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0142090320587', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-526::DEBUG::2012-12-09 12:16:03,981::task::588::TaskManager.Task::(_updateState) Task=`84b03292-fd01-4ff9-89dd-c72b5245e1cf`::moving from state preparing -> state finished
>Thread-526::DEBUG::2012-12-09 12:16:03,981::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-526::DEBUG::2012-12-09 12:16:03,981::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-526::DEBUG::2012-12-09 12:16:03,981::task::978::TaskManager.Task::(_decref) Task=`84b03292-fd01-4ff9-89dd-c72b5245e1cf`::ref 0 aborting False
>Thread-528::DEBUG::2012-12-09 12:16:05,692::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-528::DEBUG::2012-12-09 12:16:05,693::task::588::TaskManager.Task::(_updateState) Task=`602eac37-5887-49a5-ad52-d4624a1f9a07`::moving from state init -> state preparing
>Thread-528::INFO::2012-12-09 12:16:05,694::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-528::INFO::2012-12-09 12:16:05,694::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-528::DEBUG::2012-12-09 12:16:05,695::task::1172::TaskManager.Task::(prepare) Task=`602eac37-5887-49a5-ad52-d4624a1f9a07`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-528::DEBUG::2012-12-09 12:16:05,695::task::588::TaskManager.Task::(_updateState) Task=`602eac37-5887-49a5-ad52-d4624a1f9a07`::moving from state preparing -> state finished
>Thread-528::DEBUG::2012-12-09 12:16:05,696::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-528::DEBUG::2012-12-09 12:16:05,696::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-528::DEBUG::2012-12-09 12:16:05,696::task::978::TaskManager.Task::(_decref) Task=`602eac37-5887-49a5-ad52-d4624a1f9a07`::ref 0 aborting False
>Thread-529::DEBUG::2012-12-09 12:16:05,718::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-529::DEBUG::2012-12-09 12:16:05,719::task::588::TaskManager.Task::(_updateState) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::moving from state init -> state preparing
>Thread-529::INFO::2012-12-09 12:16:05,720::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-529::DEBUG::2012-12-09 12:16:05,720::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dfae5e26-8f86-431c-bbcc-2966ecc23c8e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-529::DEBUG::2012-12-09 12:16:05,720::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-529::DEBUG::2012-12-09 12:16:05,721::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-529::DEBUG::2012-12-09 12:16:05,721::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dfae5e26-8f86-431c-bbcc-2966ecc23c8e`::Granted request
>Thread-529::DEBUG::2012-12-09 12:16:05,721::task::817::TaskManager.Task::(resourceAcquired) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-529::DEBUG::2012-12-09 12:16:05,721::task::978::TaskManager.Task::(_decref) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::ref 1 aborting False
>Thread-529::DEBUG::2012-12-09 12:16:05,722::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-529::DEBUG::2012-12-09 12:16:05,722::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:05,771::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:05,796::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000653087 s, 6.3 MB/s\n'; <rc> = 0
>Thread-529::DEBUG::2012-12-09 12:16:05,832::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-529::WARNING::2012-12-09 12:16:05,834::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-529::DEBUG::2012-12-09 12:16:05,834::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-529::WARNING::2012-12-09 12:16:05,835::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-529::INFO::2012-12-09 12:16:05,836::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-529::DEBUG::2012-12-09 12:16:05,836::task::1172::TaskManager.Task::(prepare) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-529::DEBUG::2012-12-09 12:16:05,837::task::588::TaskManager.Task::(_updateState) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::moving from state preparing -> state finished
>Thread-529::DEBUG::2012-12-09 12:16:05,837::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-529::DEBUG::2012-12-09 12:16:05,838::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-529::DEBUG::2012-12-09 12:16:05,838::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-529::DEBUG::2012-12-09 12:16:05,839::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-529::DEBUG::2012-12-09 12:16:05,839::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-529::DEBUG::2012-12-09 12:16:05,840::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-529::DEBUG::2012-12-09 12:16:05,840::task::978::TaskManager.Task::(_decref) Task=`6514ddc0-1472-4edf-9662-4d7aae3063e1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:05,904::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:05,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195569 s, 52.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:07,869::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:07,870::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:07,956::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:07,974::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:07,976::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:07,976::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:07,978::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:08,003::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230319 s, 44.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:10,009::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:10,055::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229558 s, 44.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:12,062::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:12,114::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268817 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:14,121::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:14,159::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155985 s, 65.6 MB/s\n'; <rc> = 0
>Thread-534::DEBUG::2012-12-09 12:16:14,273::task::588::TaskManager.Task::(_updateState) Task=`a14f821b-6c74-45dd-8a32-7ad17b587740`::moving from state init -> state preparing
>Thread-534::INFO::2012-12-09 12:16:14,274::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-534::INFO::2012-12-09 12:16:14,274::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0251128673553', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.3', 'code': 358, 'valid': False}}
>Thread-534::DEBUG::2012-12-09 12:16:14,274::task::1172::TaskManager.Task::(prepare) Task=`a14f821b-6c74-45dd-8a32-7ad17b587740`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0251128673553', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.3', 'code': 358, 'valid': False}}
>Thread-534::DEBUG::2012-12-09 12:16:14,274::task::588::TaskManager.Task::(_updateState) Task=`a14f821b-6c74-45dd-8a32-7ad17b587740`::moving from state preparing -> state finished
>Thread-534::DEBUG::2012-12-09 12:16:14,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-534::DEBUG::2012-12-09 12:16:14,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-534::DEBUG::2012-12-09 12:16:14,275::task::978::TaskManager.Task::(_decref) Task=`a14f821b-6c74-45dd-8a32-7ad17b587740`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:16:15,804::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:15,819::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000874068 s, 4.7 MB/s\n'; <rc> = 0
>Thread-536::DEBUG::2012-12-09 12:16:15,964::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-536::DEBUG::2012-12-09 12:16:15,965::task::588::TaskManager.Task::(_updateState) Task=`80cf2d9e-3bf0-44ad-aba3-0c506a3d58c7`::moving from state init -> state preparing
>Thread-536::INFO::2012-12-09 12:16:15,966::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-536::INFO::2012-12-09 12:16:15,966::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-536::DEBUG::2012-12-09 12:16:15,967::task::1172::TaskManager.Task::(prepare) Task=`80cf2d9e-3bf0-44ad-aba3-0c506a3d58c7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-536::DEBUG::2012-12-09 12:16:15,967::task::588::TaskManager.Task::(_updateState) Task=`80cf2d9e-3bf0-44ad-aba3-0c506a3d58c7`::moving from state preparing -> state finished
>Thread-536::DEBUG::2012-12-09 12:16:15,967::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-536::DEBUG::2012-12-09 12:16:15,967::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-536::DEBUG::2012-12-09 12:16:15,967::task::978::TaskManager.Task::(_decref) Task=`80cf2d9e-3bf0-44ad-aba3-0c506a3d58c7`::ref 0 aborting False
>Thread-537::DEBUG::2012-12-09 12:16:15,989::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-537::DEBUG::2012-12-09 12:16:15,990::task::588::TaskManager.Task::(_updateState) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::moving from state init -> state preparing
>Thread-537::INFO::2012-12-09 12:16:15,990::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-537::DEBUG::2012-12-09 12:16:15,991::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`785caa55-5b51-4a97-8e34-87c9e0b56739`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-537::DEBUG::2012-12-09 12:16:15,992::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-537::DEBUG::2012-12-09 12:16:15,992::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-537::DEBUG::2012-12-09 12:16:15,993::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`785caa55-5b51-4a97-8e34-87c9e0b56739`::Granted request
>Thread-537::DEBUG::2012-12-09 12:16:15,993::task::817::TaskManager.Task::(resourceAcquired) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-537::DEBUG::2012-12-09 12:16:15,994::task::978::TaskManager.Task::(_decref) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::ref 1 aborting False
>Thread-537::DEBUG::2012-12-09 12:16:15,995::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-537::DEBUG::2012-12-09 12:16:15,996::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-537::DEBUG::2012-12-09 12:16:16,103::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-537::WARNING::2012-12-09 12:16:16,104::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-537::DEBUG::2012-12-09 12:16:16,104::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-537::WARNING::2012-12-09 12:16:16,105::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-537::INFO::2012-12-09 12:16:16,105::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-537::DEBUG::2012-12-09 12:16:16,105::task::1172::TaskManager.Task::(prepare) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-537::DEBUG::2012-12-09 12:16:16,105::task::588::TaskManager.Task::(_updateState) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::moving from state preparing -> state finished
>Thread-537::DEBUG::2012-12-09 12:16:16,106::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-537::DEBUG::2012-12-09 12:16:16,106::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-537::DEBUG::2012-12-09 12:16:16,106::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-537::DEBUG::2012-12-09 12:16:16,106::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-537::DEBUG::2012-12-09 12:16:16,107::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-537::DEBUG::2012-12-09 12:16:16,107::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-537::DEBUG::2012-12-09 12:16:16,107::task::978::TaskManager.Task::(_decref) Task=`2d26a288-3566-4ee0-81b8-fae23b39c0c7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:16,166::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:16,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240423 s, 42.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:17,979::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:17,980::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:18,104::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:18,105::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:18,105::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:18,106::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:18,222::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:18,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268873 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:20,284::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:20,335::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265216 s, 38.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:22,341::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:22,388::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215543 s, 47.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:24,394::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:24,445::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257557 s, 39.8 MB/s\n'; <rc> = 0
>Thread-542::DEBUG::2012-12-09 12:16:24,545::task::588::TaskManager.Task::(_updateState) Task=`11dea347-4b72-4e60-a0b2-5bd449733b2e`::moving from state init -> state preparing
>Thread-542::INFO::2012-12-09 12:16:24,546::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-542::INFO::2012-12-09 12:16:24,546::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158801078796', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-542::DEBUG::2012-12-09 12:16:24,547::task::1172::TaskManager.Task::(prepare) Task=`11dea347-4b72-4e60-a0b2-5bd449733b2e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158801078796', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-542::DEBUG::2012-12-09 12:16:24,547::task::588::TaskManager.Task::(_updateState) Task=`11dea347-4b72-4e60-a0b2-5bd449733b2e`::moving from state preparing -> state finished
>Thread-542::DEBUG::2012-12-09 12:16:24,548::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-542::DEBUG::2012-12-09 12:16:24,548::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-542::DEBUG::2012-12-09 12:16:24,548::task::978::TaskManager.Task::(_decref) Task=`11dea347-4b72-4e60-a0b2-5bd449733b2e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:16:25,823::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:25,834::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000531077 s, 7.7 MB/s\n'; <rc> = 0
>Thread-544::DEBUG::2012-12-09 12:16:26,224::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-544::DEBUG::2012-12-09 12:16:26,225::task::588::TaskManager.Task::(_updateState) Task=`dadeb7f7-3e6c-4523-9c33-e47828be5ef6`::moving from state init -> state preparing
>Thread-544::INFO::2012-12-09 12:16:26,226::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-544::INFO::2012-12-09 12:16:26,226::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-544::DEBUG::2012-12-09 12:16:26,227::task::1172::TaskManager.Task::(prepare) Task=`dadeb7f7-3e6c-4523-9c33-e47828be5ef6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-544::DEBUG::2012-12-09 12:16:26,227::task::588::TaskManager.Task::(_updateState) Task=`dadeb7f7-3e6c-4523-9c33-e47828be5ef6`::moving from state preparing -> state finished
>Thread-544::DEBUG::2012-12-09 12:16:26,227::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-544::DEBUG::2012-12-09 12:16:26,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-544::DEBUG::2012-12-09 12:16:26,228::task::978::TaskManager.Task::(_decref) Task=`dadeb7f7-3e6c-4523-9c33-e47828be5ef6`::ref 0 aborting False
>Thread-545::DEBUG::2012-12-09 12:16:26,252::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-545::DEBUG::2012-12-09 12:16:26,253::task::588::TaskManager.Task::(_updateState) Task=`6212096b-1948-42d3-a202-549544dbe93d`::moving from state init -> state preparing
>Thread-545::INFO::2012-12-09 12:16:26,253::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-545::DEBUG::2012-12-09 12:16:26,254::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f3a59caf-91a4-4db4-8e82-dba5b1660f95`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-545::DEBUG::2012-12-09 12:16:26,255::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-545::DEBUG::2012-12-09 12:16:26,255::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-545::DEBUG::2012-12-09 12:16:26,255::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f3a59caf-91a4-4db4-8e82-dba5b1660f95`::Granted request
>Thread-545::DEBUG::2012-12-09 12:16:26,256::task::817::TaskManager.Task::(resourceAcquired) Task=`6212096b-1948-42d3-a202-549544dbe93d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-545::DEBUG::2012-12-09 12:16:26,257::task::978::TaskManager.Task::(_decref) Task=`6212096b-1948-42d3-a202-549544dbe93d`::ref 1 aborting False
>Thread-545::DEBUG::2012-12-09 12:16:26,258::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-545::DEBUG::2012-12-09 12:16:26,259::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-545::DEBUG::2012-12-09 12:16:26,365::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-545::WARNING::2012-12-09 12:16:26,366::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-545::DEBUG::2012-12-09 12:16:26,366::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-545::WARNING::2012-12-09 12:16:26,367::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-545::INFO::2012-12-09 12:16:26,367::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-545::DEBUG::2012-12-09 12:16:26,367::task::1172::TaskManager.Task::(prepare) Task=`6212096b-1948-42d3-a202-549544dbe93d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-545::DEBUG::2012-12-09 12:16:26,367::task::588::TaskManager.Task::(_updateState) Task=`6212096b-1948-42d3-a202-549544dbe93d`::moving from state preparing -> state finished
>Thread-545::DEBUG::2012-12-09 12:16:26,367::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-545::DEBUG::2012-12-09 12:16:26,368::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-545::DEBUG::2012-12-09 12:16:26,368::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-545::DEBUG::2012-12-09 12:16:26,368::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-545::DEBUG::2012-12-09 12:16:26,368::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-545::DEBUG::2012-12-09 12:16:26,368::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-545::DEBUG::2012-12-09 12:16:26,369::task::978::TaskManager.Task::(_decref) Task=`6212096b-1948-42d3-a202-549544dbe93d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:26,452::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:26,502::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200166 s, 51.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:28,107::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:28,108::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:28,224::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:28,226::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:28,226::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:28,227::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:28,510::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:28,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191663 s, 53.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:30,562::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:30,613::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026866 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:32,620::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:32,668::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227997 s, 44.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:34,674::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:34,725::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257739 s, 39.7 MB/s\n'; <rc> = 0
>Thread-550::DEBUG::2012-12-09 12:16:34,884::task::588::TaskManager.Task::(_updateState) Task=`5f59fc2b-a3af-4a0f-9a0f-f4a4a74cbba3`::moving from state init -> state preparing
>Thread-550::INFO::2012-12-09 12:16:34,885::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-550::INFO::2012-12-09 12:16:34,886::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121901035309', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-550::DEBUG::2012-12-09 12:16:34,886::task::1172::TaskManager.Task::(prepare) Task=`5f59fc2b-a3af-4a0f-9a0f-f4a4a74cbba3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121901035309', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-550::DEBUG::2012-12-09 12:16:34,887::task::588::TaskManager.Task::(_updateState) Task=`5f59fc2b-a3af-4a0f-9a0f-f4a4a74cbba3`::moving from state preparing -> state finished
>Thread-550::DEBUG::2012-12-09 12:16:34,887::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-550::DEBUG::2012-12-09 12:16:34,887::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-550::DEBUG::2012-12-09 12:16:34,888::task::978::TaskManager.Task::(_decref) Task=`5f59fc2b-a3af-4a0f-9a0f-f4a4a74cbba3`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:16:35,840::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:35,853::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00078413 s, 5.2 MB/s\n'; <rc> = 0
>Thread-552::DEBUG::2012-12-09 12:16:36,492::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-552::DEBUG::2012-12-09 12:16:36,493::task::588::TaskManager.Task::(_updateState) Task=`f76af87e-8056-48ea-9aac-9fdf8fbb0090`::moving from state init -> state preparing
>Thread-552::INFO::2012-12-09 12:16:36,493::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-552::INFO::2012-12-09 12:16:36,494::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-552::DEBUG::2012-12-09 12:16:36,494::task::1172::TaskManager.Task::(prepare) Task=`f76af87e-8056-48ea-9aac-9fdf8fbb0090`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-552::DEBUG::2012-12-09 12:16:36,495::task::588::TaskManager.Task::(_updateState) Task=`f76af87e-8056-48ea-9aac-9fdf8fbb0090`::moving from state preparing -> state finished
>Thread-552::DEBUG::2012-12-09 12:16:36,495::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-552::DEBUG::2012-12-09 12:16:36,496::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-552::DEBUG::2012-12-09 12:16:36,496::task::978::TaskManager.Task::(_decref) Task=`f76af87e-8056-48ea-9aac-9fdf8fbb0090`::ref 0 aborting False
>Thread-553::DEBUG::2012-12-09 12:16:36,514::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-553::DEBUG::2012-12-09 12:16:36,515::task::588::TaskManager.Task::(_updateState) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::moving from state init -> state preparing
>Thread-553::INFO::2012-12-09 12:16:36,515::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-553::DEBUG::2012-12-09 12:16:36,515::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9d818406-2f83-4e02-a8e9-3cc9f7289911`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-553::DEBUG::2012-12-09 12:16:36,515::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-553::DEBUG::2012-12-09 12:16:36,516::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-553::DEBUG::2012-12-09 12:16:36,516::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9d818406-2f83-4e02-a8e9-3cc9f7289911`::Granted request
>Thread-553::DEBUG::2012-12-09 12:16:36,516::task::817::TaskManager.Task::(resourceAcquired) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-553::DEBUG::2012-12-09 12:16:36,516::task::978::TaskManager.Task::(_decref) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::ref 1 aborting False
>Thread-553::DEBUG::2012-12-09 12:16:36,517::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-553::DEBUG::2012-12-09 12:16:36,518::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-553::DEBUG::2012-12-09 12:16:36,629::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-553::WARNING::2012-12-09 12:16:36,631::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-553::DEBUG::2012-12-09 12:16:36,631::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-553::WARNING::2012-12-09 12:16:36,631::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-553::INFO::2012-12-09 12:16:36,632::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-553::DEBUG::2012-12-09 12:16:36,632::task::1172::TaskManager.Task::(prepare) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-553::DEBUG::2012-12-09 12:16:36,632::task::588::TaskManager.Task::(_updateState) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::moving from state preparing -> state finished
>Thread-553::DEBUG::2012-12-09 12:16:36,632::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-553::DEBUG::2012-12-09 12:16:36,632::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-553::DEBUG::2012-12-09 12:16:36,633::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-553::DEBUG::2012-12-09 12:16:36,633::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-553::DEBUG::2012-12-09 12:16:36,633::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-553::DEBUG::2012-12-09 12:16:36,633::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-553::DEBUG::2012-12-09 12:16:36,634::task::978::TaskManager.Task::(_decref) Task=`32e05c33-7f7c-46a1-80a9-337f822f77db`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:36,731::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:36,779::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211351 s, 48.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:38,229::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:38,230::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:38,351::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:38,352::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:38,352::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:38,352::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:38,786::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:38,823::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157695 s, 64.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:40,830::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:40,876::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216565 s, 47.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:42,884::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:42,935::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262979 s, 38.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:44,941::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:44,993::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272105 s, 37.6 MB/s\n'; <rc> = 0
>Thread-558::DEBUG::2012-12-09 12:16:45,254::task::588::TaskManager.Task::(_updateState) Task=`9af3039a-5b1c-45d3-992e-bbb9f0ec101c`::moving from state init -> state preparing
>Thread-558::INFO::2012-12-09 12:16:45,254::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-558::INFO::2012-12-09 12:16:45,255::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143351554871', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-558::DEBUG::2012-12-09 12:16:45,255::task::1172::TaskManager.Task::(prepare) Task=`9af3039a-5b1c-45d3-992e-bbb9f0ec101c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143351554871', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-558::DEBUG::2012-12-09 12:16:45,256::task::588::TaskManager.Task::(_updateState) Task=`9af3039a-5b1c-45d3-992e-bbb9f0ec101c`::moving from state preparing -> state finished
>Thread-558::DEBUG::2012-12-09 12:16:45,256::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-558::DEBUG::2012-12-09 12:16:45,257::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-558::DEBUG::2012-12-09 12:16:45,257::task::978::TaskManager.Task::(_decref) Task=`9af3039a-5b1c-45d3-992e-bbb9f0ec101c`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:16:45,859::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:45,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000917017 s, 4.5 MB/s\n'; <rc> = 0
>Thread-560::DEBUG::2012-12-09 12:16:46,810::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-560::DEBUG::2012-12-09 12:16:46,811::task::588::TaskManager.Task::(_updateState) Task=`a733213f-3b6f-4cc0-858f-61dfa4048a9e`::moving from state init -> state preparing
>Thread-560::INFO::2012-12-09 12:16:46,812::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-560::INFO::2012-12-09 12:16:46,812::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-560::DEBUG::2012-12-09 12:16:46,813::task::1172::TaskManager.Task::(prepare) Task=`a733213f-3b6f-4cc0-858f-61dfa4048a9e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-560::DEBUG::2012-12-09 12:16:46,813::task::588::TaskManager.Task::(_updateState) Task=`a733213f-3b6f-4cc0-858f-61dfa4048a9e`::moving from state preparing -> state finished
>Thread-560::DEBUG::2012-12-09 12:16:46,814::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-560::DEBUG::2012-12-09 12:16:46,814::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-560::DEBUG::2012-12-09 12:16:46,815::task::978::TaskManager.Task::(_decref) Task=`a733213f-3b6f-4cc0-858f-61dfa4048a9e`::ref 0 aborting False
>Thread-561::DEBUG::2012-12-09 12:16:46,835::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-561::DEBUG::2012-12-09 12:16:46,836::task::588::TaskManager.Task::(_updateState) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::moving from state init -> state preparing
>Thread-561::INFO::2012-12-09 12:16:46,836::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-561::DEBUG::2012-12-09 12:16:46,837::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a4840079-482b-4c8f-b715-5f53a6aa959d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-561::DEBUG::2012-12-09 12:16:46,837::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-561::DEBUG::2012-12-09 12:16:46,838::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-561::DEBUG::2012-12-09 12:16:46,838::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a4840079-482b-4c8f-b715-5f53a6aa959d`::Granted request
>Thread-561::DEBUG::2012-12-09 12:16:46,839::task::817::TaskManager.Task::(resourceAcquired) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-561::DEBUG::2012-12-09 12:16:46,840::task::978::TaskManager.Task::(_decref) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::ref 1 aborting False
>Thread-561::DEBUG::2012-12-09 12:16:46,841::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-561::DEBUG::2012-12-09 12:16:46,842::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-561::DEBUG::2012-12-09 12:16:46,957::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-561::WARNING::2012-12-09 12:16:46,959::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-561::DEBUG::2012-12-09 12:16:46,959::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-561::WARNING::2012-12-09 12:16:46,960::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-561::INFO::2012-12-09 12:16:46,961::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-561::DEBUG::2012-12-09 12:16:46,961::task::1172::TaskManager.Task::(prepare) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-561::DEBUG::2012-12-09 12:16:46,961::task::588::TaskManager.Task::(_updateState) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::moving from state preparing -> state finished
>Thread-561::DEBUG::2012-12-09 12:16:46,961::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-561::DEBUG::2012-12-09 12:16:46,961::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-561::DEBUG::2012-12-09 12:16:46,962::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-561::DEBUG::2012-12-09 12:16:46,962::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-561::DEBUG::2012-12-09 12:16:46,962::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-561::DEBUG::2012-12-09 12:16:46,962::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-561::DEBUG::2012-12-09 12:16:46,962::task::978::TaskManager.Task::(_decref) Task=`aa916158-ed0d-44bf-83aa-b1e88c61cc5f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:47,000::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:47,048::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216596 s, 47.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:48,353::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:48,355::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:48,462::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:48,463::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:48,463::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:48,464::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:49,055::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:49,104::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245124 s, 41.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:51,111::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:51,163::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271511 s, 37.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:53,170::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:53,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217712 s, 47.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:16:55,223::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:55,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268304 s, 38.2 MB/s\n'; <rc> = 0
>Thread-566::DEBUG::2012-12-09 12:16:55,559::task::588::TaskManager.Task::(_updateState) Task=`a94c6134-8f6d-4953-a195-67f00ae1e39f`::moving from state init -> state preparing
>Thread-566::INFO::2012-12-09 12:16:55,560::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-566::INFO::2012-12-09 12:16:55,560::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163290500641', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-566::DEBUG::2012-12-09 12:16:55,561::task::1172::TaskManager.Task::(prepare) Task=`a94c6134-8f6d-4953-a195-67f00ae1e39f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163290500641', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-566::DEBUG::2012-12-09 12:16:55,561::task::588::TaskManager.Task::(_updateState) Task=`a94c6134-8f6d-4953-a195-67f00ae1e39f`::moving from state preparing -> state finished
>Thread-566::DEBUG::2012-12-09 12:16:55,561::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-566::DEBUG::2012-12-09 12:16:55,561::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-566::DEBUG::2012-12-09 12:16:55,562::task::978::TaskManager.Task::(_decref) Task=`a94c6134-8f6d-4953-a195-67f00ae1e39f`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:16:55,880::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:16:55,896::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000798827 s, 5.1 MB/s\n'; <rc> = 0
>Thread-568::DEBUG::2012-12-09 12:16:57,058::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-568::DEBUG::2012-12-09 12:16:57,058::task::588::TaskManager.Task::(_updateState) Task=`a3c58b23-772b-4e25-b4c2-c19167dabcd0`::moving from state init -> state preparing
>Thread-568::INFO::2012-12-09 12:16:57,058::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-568::INFO::2012-12-09 12:16:57,059::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-568::DEBUG::2012-12-09 12:16:57,059::task::1172::TaskManager.Task::(prepare) Task=`a3c58b23-772b-4e25-b4c2-c19167dabcd0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-568::DEBUG::2012-12-09 12:16:57,059::task::588::TaskManager.Task::(_updateState) Task=`a3c58b23-772b-4e25-b4c2-c19167dabcd0`::moving from state preparing -> state finished
>Thread-568::DEBUG::2012-12-09 12:16:57,059::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-568::DEBUG::2012-12-09 12:16:57,059::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-568::DEBUG::2012-12-09 12:16:57,060::task::978::TaskManager.Task::(_decref) Task=`a3c58b23-772b-4e25-b4c2-c19167dabcd0`::ref 0 aborting False
>Thread-569::DEBUG::2012-12-09 12:16:57,072::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-569::DEBUG::2012-12-09 12:16:57,073::task::588::TaskManager.Task::(_updateState) Task=`1c423210-3fdc-4897-9581-6234420c590d`::moving from state init -> state preparing
>Thread-569::INFO::2012-12-09 12:16:57,073::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-569::DEBUG::2012-12-09 12:16:57,073::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbddf169-524d-4e61-aa01-064b67af0ce9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-569::DEBUG::2012-12-09 12:16:57,074::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-569::DEBUG::2012-12-09 12:16:57,074::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-569::DEBUG::2012-12-09 12:16:57,074::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbddf169-524d-4e61-aa01-064b67af0ce9`::Granted request
>Thread-569::DEBUG::2012-12-09 12:16:57,074::task::817::TaskManager.Task::(resourceAcquired) Task=`1c423210-3fdc-4897-9581-6234420c590d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-569::DEBUG::2012-12-09 12:16:57,075::task::978::TaskManager.Task::(_decref) Task=`1c423210-3fdc-4897-9581-6234420c590d`::ref 1 aborting False
>Thread-569::DEBUG::2012-12-09 12:16:57,075::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-569::DEBUG::2012-12-09 12:16:57,076::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-569::DEBUG::2012-12-09 12:16:57,173::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-569::WARNING::2012-12-09 12:16:57,174::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-569::DEBUG::2012-12-09 12:16:57,174::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-569::WARNING::2012-12-09 12:16:57,174::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-569::INFO::2012-12-09 12:16:57,175::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-569::DEBUG::2012-12-09 12:16:57,175::task::1172::TaskManager.Task::(prepare) Task=`1c423210-3fdc-4897-9581-6234420c590d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-569::DEBUG::2012-12-09 12:16:57,175::task::588::TaskManager.Task::(_updateState) Task=`1c423210-3fdc-4897-9581-6234420c590d`::moving from state preparing -> state finished
>Thread-569::DEBUG::2012-12-09 12:16:57,175::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-569::DEBUG::2012-12-09 12:16:57,176::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-569::DEBUG::2012-12-09 12:16:57,176::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-569::DEBUG::2012-12-09 12:16:57,176::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-569::DEBUG::2012-12-09 12:16:57,176::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-569::DEBUG::2012-12-09 12:16:57,176::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-569::DEBUG::2012-12-09 12:16:57,177::task::978::TaskManager.Task::(_decref) Task=`1c423210-3fdc-4897-9581-6234420c590d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:16:57,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:57,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016393 s, 62.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:16:58,465::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:16:58,466::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:16:58,580::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:16:58,581::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:16:58,581::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:16:58,582::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:16:59,328::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:16:59,376::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0231728 s, 44.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:01,383::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:01,444::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0295207 s, 34.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:03,451::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:03,504::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275751 s, 37.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:05,511::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:05,559::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022455 s, 45.6 MB/s\n'; <rc> = 0
>Thread-574::DEBUG::2012-12-09 12:17:05,832::task::588::TaskManager.Task::(_updateState) Task=`1afc49ef-26a2-4543-a766-215f306e80f0`::moving from state init -> state preparing
>Thread-574::INFO::2012-12-09 12:17:05,833::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-574::INFO::2012-12-09 12:17:05,834::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169529914856', 'lastCheck': '9.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-574::DEBUG::2012-12-09 12:17:05,834::task::1172::TaskManager.Task::(prepare) Task=`1afc49ef-26a2-4543-a766-215f306e80f0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169529914856', 'lastCheck': '9.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-574::DEBUG::2012-12-09 12:17:05,835::task::588::TaskManager.Task::(_updateState) Task=`1afc49ef-26a2-4543-a766-215f306e80f0`::moving from state preparing -> state finished
>Thread-574::DEBUG::2012-12-09 12:17:05,835::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-574::DEBUG::2012-12-09 12:17:05,835::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-574::DEBUG::2012-12-09 12:17:05,836::task::978::TaskManager.Task::(_decref) Task=`1afc49ef-26a2-4543-a766-215f306e80f0`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:17:05,901::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:05,917::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00101607 s, 4.0 MB/s\n'; <rc> = 0
>Thread-576::DEBUG::2012-12-09 12:17:07,273::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-576::DEBUG::2012-12-09 12:17:07,273::task::588::TaskManager.Task::(_updateState) Task=`6677eb72-a36f-4e86-abab-ece9f21df8ad`::moving from state init -> state preparing
>Thread-576::INFO::2012-12-09 12:17:07,274::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-576::INFO::2012-12-09 12:17:07,274::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-576::DEBUG::2012-12-09 12:17:07,274::task::1172::TaskManager.Task::(prepare) Task=`6677eb72-a36f-4e86-abab-ece9f21df8ad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-576::DEBUG::2012-12-09 12:17:07,274::task::588::TaskManager.Task::(_updateState) Task=`6677eb72-a36f-4e86-abab-ece9f21df8ad`::moving from state preparing -> state finished
>Thread-576::DEBUG::2012-12-09 12:17:07,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-576::DEBUG::2012-12-09 12:17:07,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-576::DEBUG::2012-12-09 12:17:07,275::task::978::TaskManager.Task::(_decref) Task=`6677eb72-a36f-4e86-abab-ece9f21df8ad`::ref 0 aborting False
>Thread-577::DEBUG::2012-12-09 12:17:07,294::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-577::DEBUG::2012-12-09 12:17:07,295::task::588::TaskManager.Task::(_updateState) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::moving from state init -> state preparing
>Thread-577::INFO::2012-12-09 12:17:07,296::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-577::DEBUG::2012-12-09 12:17:07,297::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67180f2a-5cf6-4141-83d6-8637c6be52f7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-577::DEBUG::2012-12-09 12:17:07,297::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-577::DEBUG::2012-12-09 12:17:07,297::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-577::DEBUG::2012-12-09 12:17:07,298::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67180f2a-5cf6-4141-83d6-8637c6be52f7`::Granted request
>Thread-577::DEBUG::2012-12-09 12:17:07,299::task::817::TaskManager.Task::(resourceAcquired) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-577::DEBUG::2012-12-09 12:17:07,299::task::978::TaskManager.Task::(_decref) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::ref 1 aborting False
>Thread-577::DEBUG::2012-12-09 12:17:07,300::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-577::DEBUG::2012-12-09 12:17:07,301::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-577::DEBUG::2012-12-09 12:17:07,402::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-577::WARNING::2012-12-09 12:17:07,403::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-577::DEBUG::2012-12-09 12:17:07,404::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-577::WARNING::2012-12-09 12:17:07,404::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-577::INFO::2012-12-09 12:17:07,405::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-577::DEBUG::2012-12-09 12:17:07,405::task::1172::TaskManager.Task::(prepare) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-577::DEBUG::2012-12-09 12:17:07,405::task::588::TaskManager.Task::(_updateState) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::moving from state preparing -> state finished
>Thread-577::DEBUG::2012-12-09 12:17:07,405::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-577::DEBUG::2012-12-09 12:17:07,405::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-577::DEBUG::2012-12-09 12:17:07,406::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-577::DEBUG::2012-12-09 12:17:07,406::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-577::DEBUG::2012-12-09 12:17:07,406::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-577::DEBUG::2012-12-09 12:17:07,406::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-577::DEBUG::2012-12-09 12:17:07,407::task::978::TaskManager.Task::(_decref) Task=`e28f2504-c6a0-4a3e-b409-d33dcbf538fc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:07,565::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:07,607::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176223 s, 58.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:08,582::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:08,584::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:08,695::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:08,697::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:08,697::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:08,698::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:17:09,614::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:09,657::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196355 s, 52.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:11,664::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:11,715::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260105 s, 39.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:13,722::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:13,774::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279323 s, 36.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:15,781::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:15,833::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277949 s, 36.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:17:15,922::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:15,934::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000753392 s, 5.4 MB/s\n'; <rc> = 0
>Thread-582::DEBUG::2012-12-09 12:17:16,125::task::588::TaskManager.Task::(_updateState) Task=`071841d2-3c0b-47e5-8b71-2d2b437d6c71`::moving from state init -> state preparing
>Thread-582::INFO::2012-12-09 12:17:16,125::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-582::INFO::2012-12-09 12:17:16,126::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0131020545959', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-582::DEBUG::2012-12-09 12:17:16,126::task::1172::TaskManager.Task::(prepare) Task=`071841d2-3c0b-47e5-8b71-2d2b437d6c71`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0131020545959', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-582::DEBUG::2012-12-09 12:17:16,126::task::588::TaskManager.Task::(_updateState) Task=`071841d2-3c0b-47e5-8b71-2d2b437d6c71`::moving from state preparing -> state finished
>Thread-582::DEBUG::2012-12-09 12:17:16,126::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-582::DEBUG::2012-12-09 12:17:16,126::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-582::DEBUG::2012-12-09 12:17:16,126::task::978::TaskManager.Task::(_decref) Task=`071841d2-3c0b-47e5-8b71-2d2b437d6c71`::ref 0 aborting False
>Thread-584::DEBUG::2012-12-09 12:17:17,542::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-584::DEBUG::2012-12-09 12:17:17,542::task::588::TaskManager.Task::(_updateState) Task=`4adcfbb7-f088-4508-b1d4-458d62fe0be3`::moving from state init -> state preparing
>Thread-584::INFO::2012-12-09 12:17:17,542::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-584::INFO::2012-12-09 12:17:17,543::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-584::DEBUG::2012-12-09 12:17:17,543::task::1172::TaskManager.Task::(prepare) Task=`4adcfbb7-f088-4508-b1d4-458d62fe0be3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-584::DEBUG::2012-12-09 12:17:17,543::task::588::TaskManager.Task::(_updateState) Task=`4adcfbb7-f088-4508-b1d4-458d62fe0be3`::moving from state preparing -> state finished
>Thread-584::DEBUG::2012-12-09 12:17:17,543::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-584::DEBUG::2012-12-09 12:17:17,543::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-584::DEBUG::2012-12-09 12:17:17,544::task::978::TaskManager.Task::(_decref) Task=`4adcfbb7-f088-4508-b1d4-458d62fe0be3`::ref 0 aborting False
>Thread-585::DEBUG::2012-12-09 12:17:17,568::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-585::DEBUG::2012-12-09 12:17:17,568::task::588::TaskManager.Task::(_updateState) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::moving from state init -> state preparing
>Thread-585::INFO::2012-12-09 12:17:17,568::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-585::DEBUG::2012-12-09 12:17:17,569::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`358af736-481e-4377-a023-5d31740c8f3a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-585::DEBUG::2012-12-09 12:17:17,569::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-585::DEBUG::2012-12-09 12:17:17,569::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-585::DEBUG::2012-12-09 12:17:17,569::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`358af736-481e-4377-a023-5d31740c8f3a`::Granted request
>Thread-585::DEBUG::2012-12-09 12:17:17,570::task::817::TaskManager.Task::(resourceAcquired) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-585::DEBUG::2012-12-09 12:17:17,570::task::978::TaskManager.Task::(_decref) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::ref 1 aborting False
>Thread-585::DEBUG::2012-12-09 12:17:17,571::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-585::DEBUG::2012-12-09 12:17:17,571::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-585::DEBUG::2012-12-09 12:17:17,684::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-585::WARNING::2012-12-09 12:17:17,686::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-585::DEBUG::2012-12-09 12:17:17,687::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-585::WARNING::2012-12-09 12:17:17,688::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-585::INFO::2012-12-09 12:17:17,689::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-585::DEBUG::2012-12-09 12:17:17,689::task::1172::TaskManager.Task::(prepare) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-585::DEBUG::2012-12-09 12:17:17,690::task::588::TaskManager.Task::(_updateState) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::moving from state preparing -> state finished
>Thread-585::DEBUG::2012-12-09 12:17:17,690::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-585::DEBUG::2012-12-09 12:17:17,690::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-585::DEBUG::2012-12-09 12:17:17,691::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-585::DEBUG::2012-12-09 12:17:17,691::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-585::DEBUG::2012-12-09 12:17:17,691::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-585::DEBUG::2012-12-09 12:17:17,691::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-585::DEBUG::2012-12-09 12:17:17,691::task::978::TaskManager.Task::(_decref) Task=`63aa78e1-84f4-4b2c-873c-febafc0a86cd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:17,840::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:17,892::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216394 s, 47.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:18,699::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:18,700::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:18,821::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:18,822::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:18,822::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:18,822::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:17:19,899::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:19,942::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203563 s, 50.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:21,949::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:21,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244559 s, 41.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:24,003::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:24,055::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273449 s, 37.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:17:25,940::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:25,954::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000907258 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:26,062::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:26,114::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272048 s, 37.6 MB/s\n'; <rc> = 0
>Thread-590::DEBUG::2012-12-09 12:17:26,491::task::588::TaskManager.Task::(_updateState) Task=`a4268249-295b-4eaa-9d30-281a3a86c4aa`::moving from state init -> state preparing
>Thread-590::INFO::2012-12-09 12:17:26,492::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-590::INFO::2012-12-09 12:17:26,493::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153460502625', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-590::DEBUG::2012-12-09 12:17:26,493::task::1172::TaskManager.Task::(prepare) Task=`a4268249-295b-4eaa-9d30-281a3a86c4aa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153460502625', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-590::DEBUG::2012-12-09 12:17:26,494::task::588::TaskManager.Task::(_updateState) Task=`a4268249-295b-4eaa-9d30-281a3a86c4aa`::moving from state preparing -> state finished
>Thread-590::DEBUG::2012-12-09 12:17:26,494::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-590::DEBUG::2012-12-09 12:17:26,494::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-590::DEBUG::2012-12-09 12:17:26,495::task::978::TaskManager.Task::(_decref) Task=`a4268249-295b-4eaa-9d30-281a3a86c4aa`::ref 0 aborting False
>Thread-592::DEBUG::2012-12-09 12:17:27,762::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-592::DEBUG::2012-12-09 12:17:27,763::task::588::TaskManager.Task::(_updateState) Task=`8bb0465b-1a86-471c-a886-d850f3ff75bf`::moving from state init -> state preparing
>Thread-592::INFO::2012-12-09 12:17:27,764::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-592::INFO::2012-12-09 12:17:27,764::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-592::DEBUG::2012-12-09 12:17:27,765::task::1172::TaskManager.Task::(prepare) Task=`8bb0465b-1a86-471c-a886-d850f3ff75bf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-592::DEBUG::2012-12-09 12:17:27,765::task::588::TaskManager.Task::(_updateState) Task=`8bb0465b-1a86-471c-a886-d850f3ff75bf`::moving from state preparing -> state finished
>Thread-592::DEBUG::2012-12-09 12:17:27,766::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-592::DEBUG::2012-12-09 12:17:27,766::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-592::DEBUG::2012-12-09 12:17:27,766::task::978::TaskManager.Task::(_decref) Task=`8bb0465b-1a86-471c-a886-d850f3ff75bf`::ref 0 aborting False
>Thread-593::DEBUG::2012-12-09 12:17:27,792::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-593::DEBUG::2012-12-09 12:17:27,793::task::588::TaskManager.Task::(_updateState) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::moving from state init -> state preparing
>Thread-593::INFO::2012-12-09 12:17:27,793::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-593::DEBUG::2012-12-09 12:17:27,794::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`64462168-fd92-4502-be26-a556f14668b8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-593::DEBUG::2012-12-09 12:17:27,794::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-593::DEBUG::2012-12-09 12:17:27,795::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-593::DEBUG::2012-12-09 12:17:27,795::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`64462168-fd92-4502-be26-a556f14668b8`::Granted request
>Thread-593::DEBUG::2012-12-09 12:17:27,796::task::817::TaskManager.Task::(resourceAcquired) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-593::DEBUG::2012-12-09 12:17:27,796::task::978::TaskManager.Task::(_decref) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::ref 1 aborting False
>Thread-593::DEBUG::2012-12-09 12:17:27,798::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-593::DEBUG::2012-12-09 12:17:27,799::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-593::DEBUG::2012-12-09 12:17:27,911::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-593::WARNING::2012-12-09 12:17:27,912::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-593::DEBUG::2012-12-09 12:17:27,912::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-593::WARNING::2012-12-09 12:17:27,913::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-593::INFO::2012-12-09 12:17:27,913::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-593::DEBUG::2012-12-09 12:17:27,913::task::1172::TaskManager.Task::(prepare) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-593::DEBUG::2012-12-09 12:17:27,914::task::588::TaskManager.Task::(_updateState) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::moving from state preparing -> state finished
>Thread-593::DEBUG::2012-12-09 12:17:27,914::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-593::DEBUG::2012-12-09 12:17:27,914::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-593::DEBUG::2012-12-09 12:17:27,914::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-593::DEBUG::2012-12-09 12:17:27,915::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-593::DEBUG::2012-12-09 12:17:27,915::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-593::DEBUG::2012-12-09 12:17:27,915::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-593::DEBUG::2012-12-09 12:17:27,915::task::978::TaskManager.Task::(_decref) Task=`a8c43e12-c722-41d9-a76a-f4214213b7aa`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:28,120::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:28,173::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274262 s, 37.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:28,823::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:28,825::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:28,938::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:28,939::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:28,940::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:28,940::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:17:30,179::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:30,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218975 s, 46.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:32,233::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:32,284::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260703 s, 39.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:34,291::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:34,337::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216193 s, 47.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:17:35,960::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:35,976::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000754177 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:36,344::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:36,395::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267271 s, 38.3 MB/s\n'; <rc> = 0
>Thread-598::DEBUG::2012-12-09 12:17:36,794::task::588::TaskManager.Task::(_updateState) Task=`f71b5389-2e8b-42e2-a7bf-f03dd406bf78`::moving from state init -> state preparing
>Thread-598::INFO::2012-12-09 12:17:36,794::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-598::INFO::2012-12-09 12:17:36,795::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0170888900757', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-598::DEBUG::2012-12-09 12:17:36,795::task::1172::TaskManager.Task::(prepare) Task=`f71b5389-2e8b-42e2-a7bf-f03dd406bf78`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0170888900757', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-598::DEBUG::2012-12-09 12:17:36,796::task::588::TaskManager.Task::(_updateState) Task=`f71b5389-2e8b-42e2-a7bf-f03dd406bf78`::moving from state preparing -> state finished
>Thread-598::DEBUG::2012-12-09 12:17:36,796::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-598::DEBUG::2012-12-09 12:17:36,797::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-598::DEBUG::2012-12-09 12:17:36,797::task::978::TaskManager.Task::(_decref) Task=`f71b5389-2e8b-42e2-a7bf-f03dd406bf78`::ref 0 aborting False
>Thread-600::DEBUG::2012-12-09 12:17:38,062::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-600::DEBUG::2012-12-09 12:17:38,063::task::588::TaskManager.Task::(_updateState) Task=`ac35a1c0-dcac-4e0c-b012-09a0bc35a124`::moving from state init -> state preparing
>Thread-600::INFO::2012-12-09 12:17:38,063::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-600::INFO::2012-12-09 12:17:38,064::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-600::DEBUG::2012-12-09 12:17:38,064::task::1172::TaskManager.Task::(prepare) Task=`ac35a1c0-dcac-4e0c-b012-09a0bc35a124`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-600::DEBUG::2012-12-09 12:17:38,065::task::588::TaskManager.Task::(_updateState) Task=`ac35a1c0-dcac-4e0c-b012-09a0bc35a124`::moving from state preparing -> state finished
>Thread-600::DEBUG::2012-12-09 12:17:38,065::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-600::DEBUG::2012-12-09 12:17:38,065::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-600::DEBUG::2012-12-09 12:17:38,066::task::978::TaskManager.Task::(_decref) Task=`ac35a1c0-dcac-4e0c-b012-09a0bc35a124`::ref 0 aborting False
>Thread-601::DEBUG::2012-12-09 12:17:38,085::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-601::DEBUG::2012-12-09 12:17:38,086::task::588::TaskManager.Task::(_updateState) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::moving from state init -> state preparing
>Thread-601::INFO::2012-12-09 12:17:38,087::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-601::DEBUG::2012-12-09 12:17:38,088::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fa069f74-3cd1-450b-8d24-1266e64bd458`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-601::DEBUG::2012-12-09 12:17:38,088::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-601::DEBUG::2012-12-09 12:17:38,089::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-601::DEBUG::2012-12-09 12:17:38,089::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fa069f74-3cd1-450b-8d24-1266e64bd458`::Granted request
>Thread-601::DEBUG::2012-12-09 12:17:38,090::task::817::TaskManager.Task::(resourceAcquired) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-601::DEBUG::2012-12-09 12:17:38,090::task::978::TaskManager.Task::(_decref) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::ref 1 aborting False
>Thread-601::DEBUG::2012-12-09 12:17:38,091::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-601::DEBUG::2012-12-09 12:17:38,091::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-601::DEBUG::2012-12-09 12:17:38,200::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-601::WARNING::2012-12-09 12:17:38,201::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-601::DEBUG::2012-12-09 12:17:38,201::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-601::WARNING::2012-12-09 12:17:38,202::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-601::INFO::2012-12-09 12:17:38,202::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-601::DEBUG::2012-12-09 12:17:38,202::task::1172::TaskManager.Task::(prepare) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-601::DEBUG::2012-12-09 12:17:38,203::task::588::TaskManager.Task::(_updateState) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::moving from state preparing -> state finished
>Thread-601::DEBUG::2012-12-09 12:17:38,203::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-601::DEBUG::2012-12-09 12:17:38,203::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-601::DEBUG::2012-12-09 12:17:38,203::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-601::DEBUG::2012-12-09 12:17:38,203::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-601::DEBUG::2012-12-09 12:17:38,203::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-601::DEBUG::2012-12-09 12:17:38,204::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-601::DEBUG::2012-12-09 12:17:38,204::task::978::TaskManager.Task::(_decref) Task=`40735a69-26e5-4494-9b8f-9c029e2c58cf`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:38,401::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:38,447::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211814 s, 48.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:38,941::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:38,943::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:39,061::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:39,062::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:39,062::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:39,063::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:17:40,455::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:40,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183259 s, 55.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:42,505::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:42,543::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156855 s, 65.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:44,549::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:44,591::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157785 s, 64.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:17:45,981::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:45,994::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000764841 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:46,601::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:46,641::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165793 s, 61.8 MB/s\n'; <rc> = 0
>Thread-606::DEBUG::2012-12-09 12:17:46,999::task::588::TaskManager.Task::(_updateState) Task=`8c9d2a3e-096c-450f-a0ef-462b1786382d`::moving from state init -> state preparing
>Thread-606::INFO::2012-12-09 12:17:46,999::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-606::INFO::2012-12-09 12:17:46,999::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138430595398', 'lastCheck': '1.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-606::DEBUG::2012-12-09 12:17:46,999::task::1172::TaskManager.Task::(prepare) Task=`8c9d2a3e-096c-450f-a0ef-462b1786382d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138430595398', 'lastCheck': '1.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-606::DEBUG::2012-12-09 12:17:47,000::task::588::TaskManager.Task::(_updateState) Task=`8c9d2a3e-096c-450f-a0ef-462b1786382d`::moving from state preparing -> state finished
>Thread-606::DEBUG::2012-12-09 12:17:47,000::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-606::DEBUG::2012-12-09 12:17:47,000::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-606::DEBUG::2012-12-09 12:17:47,000::task::978::TaskManager.Task::(_decref) Task=`8c9d2a3e-096c-450f-a0ef-462b1786382d`::ref 0 aborting False
>Thread-608::DEBUG::2012-12-09 12:17:48,324::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-608::DEBUG::2012-12-09 12:17:48,325::task::588::TaskManager.Task::(_updateState) Task=`305aa4da-8677-4aa2-978c-91496a8ff141`::moving from state init -> state preparing
>Thread-608::INFO::2012-12-09 12:17:48,325::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-608::INFO::2012-12-09 12:17:48,325::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-608::DEBUG::2012-12-09 12:17:48,326::task::1172::TaskManager.Task::(prepare) Task=`305aa4da-8677-4aa2-978c-91496a8ff141`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-608::DEBUG::2012-12-09 12:17:48,326::task::588::TaskManager.Task::(_updateState) Task=`305aa4da-8677-4aa2-978c-91496a8ff141`::moving from state preparing -> state finished
>Thread-608::DEBUG::2012-12-09 12:17:48,326::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-608::DEBUG::2012-12-09 12:17:48,326::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-608::DEBUG::2012-12-09 12:17:48,326::task::978::TaskManager.Task::(_decref) Task=`305aa4da-8677-4aa2-978c-91496a8ff141`::ref 0 aborting False
>Thread-609::DEBUG::2012-12-09 12:17:48,338::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-609::DEBUG::2012-12-09 12:17:48,339::task::588::TaskManager.Task::(_updateState) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::moving from state init -> state preparing
>Thread-609::INFO::2012-12-09 12:17:48,339::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-609::DEBUG::2012-12-09 12:17:48,339::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f68d3c78-8870-461d-af76-6b7ea87fd569`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-609::DEBUG::2012-12-09 12:17:48,339::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-609::DEBUG::2012-12-09 12:17:48,340::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-609::DEBUG::2012-12-09 12:17:48,340::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f68d3c78-8870-461d-af76-6b7ea87fd569`::Granted request
>Thread-609::DEBUG::2012-12-09 12:17:48,341::task::817::TaskManager.Task::(resourceAcquired) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-609::DEBUG::2012-12-09 12:17:48,341::task::978::TaskManager.Task::(_decref) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::ref 1 aborting False
>Thread-609::DEBUG::2012-12-09 12:17:48,341::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-609::DEBUG::2012-12-09 12:17:48,342::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-609::DEBUG::2012-12-09 12:17:48,449::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-609::WARNING::2012-12-09 12:17:48,451::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-609::DEBUG::2012-12-09 12:17:48,451::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-609::WARNING::2012-12-09 12:17:48,452::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-609::INFO::2012-12-09 12:17:48,452::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-609::DEBUG::2012-12-09 12:17:48,452::task::1172::TaskManager.Task::(prepare) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-609::DEBUG::2012-12-09 12:17:48,452::task::588::TaskManager.Task::(_updateState) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::moving from state preparing -> state finished
>Thread-609::DEBUG::2012-12-09 12:17:48,452::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-609::DEBUG::2012-12-09 12:17:48,453::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-609::DEBUG::2012-12-09 12:17:48,453::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-609::DEBUG::2012-12-09 12:17:48,453::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-609::DEBUG::2012-12-09 12:17:48,453::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-609::DEBUG::2012-12-09 12:17:48,453::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-609::DEBUG::2012-12-09 12:17:48,454::task::978::TaskManager.Task::(_decref) Task=`681cf54c-ff27-4960-b0eb-f812b777de87`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:48,647::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:48,685::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151888 s, 67.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:49,064::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:49,064::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:49,160::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:49,161::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:49,161::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:49,161::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:17:50,692::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:50,730::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156418 s, 65.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:52,736::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:52,774::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162644 s, 63.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:54,781::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:54,833::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283722 s, 36.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:17:56,000::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:17:56,015::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000901898 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:17:56,840::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:56,886::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235469 s, 43.5 MB/s\n'; <rc> = 0
>Thread-614::DEBUG::2012-12-09 12:17:57,574::task::588::TaskManager.Task::(_updateState) Task=`c912cb40-3962-4fb3-8c02-54a07f337b86`::moving from state init -> state preparing
>Thread-614::INFO::2012-12-09 12:17:57,575::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-614::INFO::2012-12-09 12:17:57,576::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.014927148819', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-614::DEBUG::2012-12-09 12:17:57,576::task::1172::TaskManager.Task::(prepare) Task=`c912cb40-3962-4fb3-8c02-54a07f337b86`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.014927148819', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-614::DEBUG::2012-12-09 12:17:57,577::task::588::TaskManager.Task::(_updateState) Task=`c912cb40-3962-4fb3-8c02-54a07f337b86`::moving from state preparing -> state finished
>Thread-614::DEBUG::2012-12-09 12:17:57,577::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-614::DEBUG::2012-12-09 12:17:57,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-614::DEBUG::2012-12-09 12:17:57,578::task::978::TaskManager.Task::(_decref) Task=`c912cb40-3962-4fb3-8c02-54a07f337b86`::ref 0 aborting False
>Thread-616::DEBUG::2012-12-09 12:17:58,583::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-616::DEBUG::2012-12-09 12:17:58,583::task::588::TaskManager.Task::(_updateState) Task=`3aacbac8-d0ef-4604-94a5-5fff626fc0bf`::moving from state init -> state preparing
>Thread-616::INFO::2012-12-09 12:17:58,584::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-616::INFO::2012-12-09 12:17:58,584::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-616::DEBUG::2012-12-09 12:17:58,584::task::1172::TaskManager.Task::(prepare) Task=`3aacbac8-d0ef-4604-94a5-5fff626fc0bf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-616::DEBUG::2012-12-09 12:17:58,584::task::588::TaskManager.Task::(_updateState) Task=`3aacbac8-d0ef-4604-94a5-5fff626fc0bf`::moving from state preparing -> state finished
>Thread-616::DEBUG::2012-12-09 12:17:58,584::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-616::DEBUG::2012-12-09 12:17:58,585::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-616::DEBUG::2012-12-09 12:17:58,585::task::978::TaskManager.Task::(_decref) Task=`3aacbac8-d0ef-4604-94a5-5fff626fc0bf`::ref 0 aborting False
>Thread-617::DEBUG::2012-12-09 12:17:58,611::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-617::DEBUG::2012-12-09 12:17:58,612::task::588::TaskManager.Task::(_updateState) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::moving from state init -> state preparing
>Thread-617::INFO::2012-12-09 12:17:58,613::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-617::DEBUG::2012-12-09 12:17:58,614::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e290d2f2-7cc7-43a7-9c1d-1a8514ed2584`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-617::DEBUG::2012-12-09 12:17:58,614::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-617::DEBUG::2012-12-09 12:17:58,615::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-617::DEBUG::2012-12-09 12:17:58,615::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e290d2f2-7cc7-43a7-9c1d-1a8514ed2584`::Granted request
>Thread-617::DEBUG::2012-12-09 12:17:58,615::task::817::TaskManager.Task::(resourceAcquired) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-617::DEBUG::2012-12-09 12:17:58,616::task::978::TaskManager.Task::(_decref) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::ref 1 aborting False
>Thread-617::DEBUG::2012-12-09 12:17:58,617::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-617::DEBUG::2012-12-09 12:17:58,617::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-617::DEBUG::2012-12-09 12:17:58,724::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-617::WARNING::2012-12-09 12:17:58,726::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-617::DEBUG::2012-12-09 12:17:58,727::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-617::WARNING::2012-12-09 12:17:58,727::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-617::INFO::2012-12-09 12:17:58,728::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-617::DEBUG::2012-12-09 12:17:58,729::task::1172::TaskManager.Task::(prepare) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-617::DEBUG::2012-12-09 12:17:58,729::task::588::TaskManager.Task::(_updateState) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::moving from state preparing -> state finished
>Thread-617::DEBUG::2012-12-09 12:17:58,729::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-617::DEBUG::2012-12-09 12:17:58,730::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-617::DEBUG::2012-12-09 12:17:58,730::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-617::DEBUG::2012-12-09 12:17:58,730::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-617::DEBUG::2012-12-09 12:17:58,731::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-617::DEBUG::2012-12-09 12:17:58,731::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-617::DEBUG::2012-12-09 12:17:58,731::task::978::TaskManager.Task::(_decref) Task=`115ce2dc-363b-458a-b697-9403cbecaec6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:17:58,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:17:58,938::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203028 s, 50.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:17:59,163::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:17:59,164::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:17:59,285::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:17:59,287::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:17:59,288::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:17:59,289::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:18:00,945::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:00,988::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196988 s, 52.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:02,995::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:03,039::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019608 s, 52.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:05,046::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:05,091::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186486 s, 54.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:06,018::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:06,045::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000925872 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:07,102::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:07,153::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263662 s, 38.8 MB/s\n'; <rc> = 0
>Thread-622::DEBUG::2012-12-09 12:18:08,028::task::588::TaskManager.Task::(_updateState) Task=`7c0c31bf-d0d6-4ac0-8af6-44d9a2dbe786`::moving from state init -> state preparing
>Thread-622::INFO::2012-12-09 12:18:08,028::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-622::INFO::2012-12-09 12:18:08,028::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0277280807495', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-622::DEBUG::2012-12-09 12:18:08,028::task::1172::TaskManager.Task::(prepare) Task=`7c0c31bf-d0d6-4ac0-8af6-44d9a2dbe786`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0277280807495', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-622::DEBUG::2012-12-09 12:18:08,029::task::588::TaskManager.Task::(_updateState) Task=`7c0c31bf-d0d6-4ac0-8af6-44d9a2dbe786`::moving from state preparing -> state finished
>Thread-622::DEBUG::2012-12-09 12:18:08,029::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-622::DEBUG::2012-12-09 12:18:08,029::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-622::DEBUG::2012-12-09 12:18:08,029::task::978::TaskManager.Task::(_decref) Task=`7c0c31bf-d0d6-4ac0-8af6-44d9a2dbe786`::ref 0 aborting False
>Thread-624::DEBUG::2012-12-09 12:18:08,835::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-624::DEBUG::2012-12-09 12:18:08,836::task::588::TaskManager.Task::(_updateState) Task=`d9d91bbf-2caa-4c94-8019-505ba2c085f0`::moving from state init -> state preparing
>Thread-624::INFO::2012-12-09 12:18:08,836::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-624::INFO::2012-12-09 12:18:08,837::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-624::DEBUG::2012-12-09 12:18:08,837::task::1172::TaskManager.Task::(prepare) Task=`d9d91bbf-2caa-4c94-8019-505ba2c085f0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-624::DEBUG::2012-12-09 12:18:08,838::task::588::TaskManager.Task::(_updateState) Task=`d9d91bbf-2caa-4c94-8019-505ba2c085f0`::moving from state preparing -> state finished
>Thread-624::DEBUG::2012-12-09 12:18:08,838::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-624::DEBUG::2012-12-09 12:18:08,838::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-624::DEBUG::2012-12-09 12:18:08,839::task::978::TaskManager.Task::(_decref) Task=`d9d91bbf-2caa-4c94-8019-505ba2c085f0`::ref 0 aborting False
>Thread-625::DEBUG::2012-12-09 12:18:08,855::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-625::DEBUG::2012-12-09 12:18:08,856::task::588::TaskManager.Task::(_updateState) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::moving from state init -> state preparing
>Thread-625::INFO::2012-12-09 12:18:08,856::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-625::DEBUG::2012-12-09 12:18:08,857::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`81fa7891-3633-4877-959f-40b8b2462ce1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-625::DEBUG::2012-12-09 12:18:08,857::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-625::DEBUG::2012-12-09 12:18:08,858::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-625::DEBUG::2012-12-09 12:18:08,858::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`81fa7891-3633-4877-959f-40b8b2462ce1`::Granted request
>Thread-625::DEBUG::2012-12-09 12:18:08,859::task::817::TaskManager.Task::(resourceAcquired) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-625::DEBUG::2012-12-09 12:18:08,859::task::978::TaskManager.Task::(_decref) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::ref 1 aborting False
>Thread-625::DEBUG::2012-12-09 12:18:08,860::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-625::DEBUG::2012-12-09 12:18:08,861::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-625::DEBUG::2012-12-09 12:18:08,968::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-625::WARNING::2012-12-09 12:18:08,970::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-625::DEBUG::2012-12-09 12:18:08,970::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-625::WARNING::2012-12-09 12:18:08,971::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-625::INFO::2012-12-09 12:18:08,971::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-625::DEBUG::2012-12-09 12:18:08,971::task::1172::TaskManager.Task::(prepare) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-625::DEBUG::2012-12-09 12:18:08,972::task::588::TaskManager.Task::(_updateState) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::moving from state preparing -> state finished
>Thread-625::DEBUG::2012-12-09 12:18:08,972::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-625::DEBUG::2012-12-09 12:18:08,972::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-625::DEBUG::2012-12-09 12:18:08,972::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-625::DEBUG::2012-12-09 12:18:08,972::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-625::DEBUG::2012-12-09 12:18:08,973::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-625::DEBUG::2012-12-09 12:18:08,973::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-625::DEBUG::2012-12-09 12:18:08,973::task::978::TaskManager.Task::(_decref) Task=`fb5b1f4a-1b3c-48b3-8cef-9fe74ea88033`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:18:09,160::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:09,208::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216206 s, 47.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:18:09,290::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:09,291::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:18:09,410::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:18:09,411::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:18:09,411::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:18:09,412::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:18:11,214::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:11,265::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257045 s, 39.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:13,271::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:13,310::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164918 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:15,317::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:15,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247601 s, 41.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:16,048::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:16,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000891194 s, 4.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:17,372::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:17,424::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270754 s, 37.8 MB/s\n'; <rc> = 0
>Thread-630::DEBUG::2012-12-09 12:18:18,324::task::588::TaskManager.Task::(_updateState) Task=`e96ee6ff-87d0-4c00-8c19-aaa478eaac02`::moving from state init -> state preparing
>Thread-630::INFO::2012-12-09 12:18:18,325::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-630::INFO::2012-12-09 12:18:18,326::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168080329895', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-630::DEBUG::2012-12-09 12:18:18,326::task::1172::TaskManager.Task::(prepare) Task=`e96ee6ff-87d0-4c00-8c19-aaa478eaac02`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168080329895', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-630::DEBUG::2012-12-09 12:18:18,326::task::588::TaskManager.Task::(_updateState) Task=`e96ee6ff-87d0-4c00-8c19-aaa478eaac02`::moving from state preparing -> state finished
>Thread-630::DEBUG::2012-12-09 12:18:18,327::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-630::DEBUG::2012-12-09 12:18:18,327::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-630::DEBUG::2012-12-09 12:18:18,328::task::978::TaskManager.Task::(_decref) Task=`e96ee6ff-87d0-4c00-8c19-aaa478eaac02`::ref 0 aborting False
>Thread-632::DEBUG::2012-12-09 12:18:19,122::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-632::DEBUG::2012-12-09 12:18:19,123::task::588::TaskManager.Task::(_updateState) Task=`287d42c7-396f-4188-a988-885df9c4e7ed`::moving from state init -> state preparing
>Thread-632::INFO::2012-12-09 12:18:19,123::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-632::INFO::2012-12-09 12:18:19,124::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-632::DEBUG::2012-12-09 12:18:19,124::task::1172::TaskManager.Task::(prepare) Task=`287d42c7-396f-4188-a988-885df9c4e7ed`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-632::DEBUG::2012-12-09 12:18:19,125::task::588::TaskManager.Task::(_updateState) Task=`287d42c7-396f-4188-a988-885df9c4e7ed`::moving from state preparing -> state finished
>Thread-632::DEBUG::2012-12-09 12:18:19,125::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-632::DEBUG::2012-12-09 12:18:19,126::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-632::DEBUG::2012-12-09 12:18:19,126::task::978::TaskManager.Task::(_decref) Task=`287d42c7-396f-4188-a988-885df9c4e7ed`::ref 0 aborting False
>Thread-633::DEBUG::2012-12-09 12:18:19,150::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-633::DEBUG::2012-12-09 12:18:19,151::task::588::TaskManager.Task::(_updateState) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::moving from state init -> state preparing
>Thread-633::INFO::2012-12-09 12:18:19,151::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-633::DEBUG::2012-12-09 12:18:19,152::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e68a4343-419b-433d-bf22-0247fdbe4acc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-633::DEBUG::2012-12-09 12:18:19,152::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-633::DEBUG::2012-12-09 12:18:19,153::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-633::DEBUG::2012-12-09 12:18:19,153::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e68a4343-419b-433d-bf22-0247fdbe4acc`::Granted request
>Thread-633::DEBUG::2012-12-09 12:18:19,154::task::817::TaskManager.Task::(resourceAcquired) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-633::DEBUG::2012-12-09 12:18:19,154::task::978::TaskManager.Task::(_decref) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::ref 1 aborting False
>Thread-633::DEBUG::2012-12-09 12:18:19,156::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-633::DEBUG::2012-12-09 12:18:19,157::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-633::DEBUG::2012-12-09 12:18:19,272::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-633::WARNING::2012-12-09 12:18:19,274::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-633::DEBUG::2012-12-09 12:18:19,274::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-633::WARNING::2012-12-09 12:18:19,275::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-633::INFO::2012-12-09 12:18:19,276::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-633::DEBUG::2012-12-09 12:18:19,276::task::1172::TaskManager.Task::(prepare) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-633::DEBUG::2012-12-09 12:18:19,277::task::588::TaskManager.Task::(_updateState) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::moving from state preparing -> state finished
>Thread-633::DEBUG::2012-12-09 12:18:19,277::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-633::DEBUG::2012-12-09 12:18:19,278::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-633::DEBUG::2012-12-09 12:18:19,278::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-633::DEBUG::2012-12-09 12:18:19,279::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-633::DEBUG::2012-12-09 12:18:19,279::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-633::DEBUG::2012-12-09 12:18:19,280::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-633::DEBUG::2012-12-09 12:18:19,280::task::978::TaskManager.Task::(_decref) Task=`dab1f291-2ff4-49c8-aa61-12305a5fa671`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:19,413::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:19,414::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:19,431::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:19,470::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169989 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:18:19,527::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:18:19,529::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:18:19,529::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:18:19,530::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:18:21,477::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:21,520::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194953 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:23,526::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:23,593::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0431358 s, 23.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:25,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:25,649::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212749 s, 48.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:26,070::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:26,093::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000875347 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:27,656::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:27,707::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256214 s, 40.0 MB/s\n'; <rc> = 0
>Thread-638::DEBUG::2012-12-09 12:18:28,606::task::588::TaskManager.Task::(_updateState) Task=`456f67b2-6753-47c8-ab5a-532837c4d375`::moving from state init -> state preparing
>Thread-638::INFO::2012-12-09 12:18:28,607::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-638::INFO::2012-12-09 12:18:28,607::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0241029262543', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-638::DEBUG::2012-12-09 12:18:28,608::task::1172::TaskManager.Task::(prepare) Task=`456f67b2-6753-47c8-ab5a-532837c4d375`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0241029262543', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-638::DEBUG::2012-12-09 12:18:28,608::task::588::TaskManager.Task::(_updateState) Task=`456f67b2-6753-47c8-ab5a-532837c4d375`::moving from state preparing -> state finished
>Thread-638::DEBUG::2012-12-09 12:18:28,609::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-638::DEBUG::2012-12-09 12:18:28,609::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-638::DEBUG::2012-12-09 12:18:28,610::task::978::TaskManager.Task::(_decref) Task=`456f67b2-6753-47c8-ab5a-532837c4d375`::ref 0 aborting False
>Thread-640::DEBUG::2012-12-09 12:18:29,405::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-640::DEBUG::2012-12-09 12:18:29,406::task::588::TaskManager.Task::(_updateState) Task=`320aa0f6-863e-4f49-add4-3b386ec45a21`::moving from state init -> state preparing
>Thread-640::INFO::2012-12-09 12:18:29,406::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-640::INFO::2012-12-09 12:18:29,407::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-640::DEBUG::2012-12-09 12:18:29,407::task::1172::TaskManager.Task::(prepare) Task=`320aa0f6-863e-4f49-add4-3b386ec45a21`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-640::DEBUG::2012-12-09 12:18:29,408::task::588::TaskManager.Task::(_updateState) Task=`320aa0f6-863e-4f49-add4-3b386ec45a21`::moving from state preparing -> state finished
>Thread-640::DEBUG::2012-12-09 12:18:29,408::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-640::DEBUG::2012-12-09 12:18:29,409::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-640::DEBUG::2012-12-09 12:18:29,409::task::978::TaskManager.Task::(_decref) Task=`320aa0f6-863e-4f49-add4-3b386ec45a21`::ref 0 aborting False
>Thread-641::DEBUG::2012-12-09 12:18:29,426::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-641::DEBUG::2012-12-09 12:18:29,427::task::588::TaskManager.Task::(_updateState) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::moving from state init -> state preparing
>Thread-641::INFO::2012-12-09 12:18:29,427::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-641::DEBUG::2012-12-09 12:18:29,428::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0bc57ecb-02e7-418e-9a09-173ad02f32ec`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-641::DEBUG::2012-12-09 12:18:29,428::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-641::DEBUG::2012-12-09 12:18:29,429::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-641::DEBUG::2012-12-09 12:18:29,429::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0bc57ecb-02e7-418e-9a09-173ad02f32ec`::Granted request
>Thread-641::DEBUG::2012-12-09 12:18:29,430::task::817::TaskManager.Task::(resourceAcquired) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-641::DEBUG::2012-12-09 12:18:29,430::task::978::TaskManager.Task::(_decref) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::ref 1 aborting False
>Thread-641::DEBUG::2012-12-09 12:18:29,432::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-641::DEBUG::2012-12-09 12:18:29,433::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-641::DEBUG::2012-12-09 12:18:29,551::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-641::WARNING::2012-12-09 12:18:29,553::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-641::DEBUG::2012-12-09 12:18:29,554::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:29,555::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:29,556::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-641::WARNING::2012-12-09 12:18:29,562::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-641::INFO::2012-12-09 12:18:29,563::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-641::DEBUG::2012-12-09 12:18:29,563::task::1172::TaskManager.Task::(prepare) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-641::DEBUG::2012-12-09 12:18:29,563::task::588::TaskManager.Task::(_updateState) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::moving from state preparing -> state finished
>Thread-641::DEBUG::2012-12-09 12:18:29,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-641::DEBUG::2012-12-09 12:18:29,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-641::DEBUG::2012-12-09 12:18:29,565::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-641::DEBUG::2012-12-09 12:18:29,565::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-641::DEBUG::2012-12-09 12:18:29,566::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-641::DEBUG::2012-12-09 12:18:29,569::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-641::DEBUG::2012-12-09 12:18:29,570::task::978::TaskManager.Task::(_decref) Task=`0291cfe0-f8a4-4580-aff1-86e655cec764`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:29,668::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:18:29,670::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:18:29,670::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:18:29,671::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:18:29,714::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:29,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0293093 s, 34.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:31,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:31,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204873 s, 50.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:33,824::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:33,875::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260928 s, 39.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:35,882::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:35,929::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224756 s, 45.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:36,096::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:36,111::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000970548 s, 4.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:37,936::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:37,985::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249455 s, 41.0 MB/s\n'; <rc> = 0
>Thread-646::DEBUG::2012-12-09 12:18:38,875::task::588::TaskManager.Task::(_updateState) Task=`eb0e1c7c-39a7-420e-b093-2b25fd7b099e`::moving from state init -> state preparing
>Thread-646::INFO::2012-12-09 12:18:38,876::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-646::INFO::2012-12-09 12:18:38,876::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0148141384125', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-646::DEBUG::2012-12-09 12:18:38,877::task::1172::TaskManager.Task::(prepare) Task=`eb0e1c7c-39a7-420e-b093-2b25fd7b099e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0148141384125', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-646::DEBUG::2012-12-09 12:18:38,877::task::588::TaskManager.Task::(_updateState) Task=`eb0e1c7c-39a7-420e-b093-2b25fd7b099e`::moving from state preparing -> state finished
>Thread-646::DEBUG::2012-12-09 12:18:38,878::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-646::DEBUG::2012-12-09 12:18:38,878::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-646::DEBUG::2012-12-09 12:18:38,878::task::978::TaskManager.Task::(_decref) Task=`eb0e1c7c-39a7-420e-b093-2b25fd7b099e`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:39,672::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:39,673::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-648::DEBUG::2012-12-09 12:18:39,762::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-648::DEBUG::2012-12-09 12:18:39,763::task::588::TaskManager.Task::(_updateState) Task=`af2afd70-31bd-490f-82cb-78cb176a30a7`::moving from state init -> state preparing
>Thread-648::INFO::2012-12-09 12:18:39,763::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-648::INFO::2012-12-09 12:18:39,763::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-648::DEBUG::2012-12-09 12:18:39,764::task::1172::TaskManager.Task::(prepare) Task=`af2afd70-31bd-490f-82cb-78cb176a30a7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-648::DEBUG::2012-12-09 12:18:39,764::task::588::TaskManager.Task::(_updateState) Task=`af2afd70-31bd-490f-82cb-78cb176a30a7`::moving from state preparing -> state finished
>Thread-648::DEBUG::2012-12-09 12:18:39,764::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-648::DEBUG::2012-12-09 12:18:39,764::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-648::DEBUG::2012-12-09 12:18:39,764::task::978::TaskManager.Task::(_decref) Task=`af2afd70-31bd-490f-82cb-78cb176a30a7`::ref 0 aborting False
>Thread-649::DEBUG::2012-12-09 12:18:39,777::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-649::DEBUG::2012-12-09 12:18:39,778::task::588::TaskManager.Task::(_updateState) Task=`a621254d-1281-4056-92f8-e83b711d8089`::moving from state init -> state preparing
>Thread-649::INFO::2012-12-09 12:18:39,778::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-649::DEBUG::2012-12-09 12:18:39,779::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de24e3b0-cb61-4ac4-a51a-dcbf0e6d1fb3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-649::DEBUG::2012-12-09 12:18:39,779::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-649::DEBUG::2012-12-09 12:18:39,779::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-649::DEBUG::2012-12-09 12:18:39,779::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de24e3b0-cb61-4ac4-a51a-dcbf0e6d1fb3`::Granted request
>Thread-649::DEBUG::2012-12-09 12:18:39,780::task::817::TaskManager.Task::(resourceAcquired) Task=`a621254d-1281-4056-92f8-e83b711d8089`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-649::DEBUG::2012-12-09 12:18:39,780::task::978::TaskManager.Task::(_decref) Task=`a621254d-1281-4056-92f8-e83b711d8089`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:39,787::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:18:39,788::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:18:39,788::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:18:39,789::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-649::DEBUG::2012-12-09 12:18:39,789::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-649::DEBUG::2012-12-09 12:18:39,790::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-649::DEBUG::2012-12-09 12:18:39,901::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-649::WARNING::2012-12-09 12:18:39,902::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-649::DEBUG::2012-12-09 12:18:39,903::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-649::WARNING::2012-12-09 12:18:39,903::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-649::INFO::2012-12-09 12:18:39,903::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-649::DEBUG::2012-12-09 12:18:39,904::task::1172::TaskManager.Task::(prepare) Task=`a621254d-1281-4056-92f8-e83b711d8089`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-649::DEBUG::2012-12-09 12:18:39,904::task::588::TaskManager.Task::(_updateState) Task=`a621254d-1281-4056-92f8-e83b711d8089`::moving from state preparing -> state finished
>Thread-649::DEBUG::2012-12-09 12:18:39,904::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-649::DEBUG::2012-12-09 12:18:39,904::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-649::DEBUG::2012-12-09 12:18:39,905::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-649::DEBUG::2012-12-09 12:18:39,905::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-649::DEBUG::2012-12-09 12:18:39,905::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-649::DEBUG::2012-12-09 12:18:39,905::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-649::DEBUG::2012-12-09 12:18:39,905::task::978::TaskManager.Task::(_decref) Task=`a621254d-1281-4056-92f8-e83b711d8089`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:18:39,991::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:40,044::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276463 s, 37.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:42,051::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:42,098::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218361 s, 46.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:44,104::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:44,155::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261582 s, 39.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:46,114::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:46,126::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00100665 s, 4.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:46,162::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:46,215::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270054 s, 37.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:48,221::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:48,269::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223529 s, 45.8 MB/s\n'; <rc> = 0
>Thread-654::DEBUG::2012-12-09 12:18:49,189::task::588::TaskManager.Task::(_updateState) Task=`5df02293-4663-4796-87db-4cbf9258ee73`::moving from state init -> state preparing
>Thread-654::INFO::2012-12-09 12:18:49,189::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-654::INFO::2012-12-09 12:18:49,189::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135440826416', 'lastCheck': '3.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-654::DEBUG::2012-12-09 12:18:49,189::task::1172::TaskManager.Task::(prepare) Task=`5df02293-4663-4796-87db-4cbf9258ee73`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135440826416', 'lastCheck': '3.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-654::DEBUG::2012-12-09 12:18:49,190::task::588::TaskManager.Task::(_updateState) Task=`5df02293-4663-4796-87db-4cbf9258ee73`::moving from state preparing -> state finished
>Thread-654::DEBUG::2012-12-09 12:18:49,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-654::DEBUG::2012-12-09 12:18:49,190::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-654::DEBUG::2012-12-09 12:18:49,190::task::978::TaskManager.Task::(_decref) Task=`5df02293-4663-4796-87db-4cbf9258ee73`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:49,790::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:49,791::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:18:49,903::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:18:49,905::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:18:49,906::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:18:49,907::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-656::DEBUG::2012-12-09 12:18:50,046::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-656::DEBUG::2012-12-09 12:18:50,047::task::588::TaskManager.Task::(_updateState) Task=`c20cc6cc-288e-49a2-83ad-155ad41c5d2a`::moving from state init -> state preparing
>Thread-656::INFO::2012-12-09 12:18:50,048::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-656::INFO::2012-12-09 12:18:50,048::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-656::DEBUG::2012-12-09 12:18:50,049::task::1172::TaskManager.Task::(prepare) Task=`c20cc6cc-288e-49a2-83ad-155ad41c5d2a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-656::DEBUG::2012-12-09 12:18:50,049::task::588::TaskManager.Task::(_updateState) Task=`c20cc6cc-288e-49a2-83ad-155ad41c5d2a`::moving from state preparing -> state finished
>Thread-656::DEBUG::2012-12-09 12:18:50,050::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-656::DEBUG::2012-12-09 12:18:50,050::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-656::DEBUG::2012-12-09 12:18:50,050::task::978::TaskManager.Task::(_decref) Task=`c20cc6cc-288e-49a2-83ad-155ad41c5d2a`::ref 0 aborting False
>Thread-657::DEBUG::2012-12-09 12:18:50,074::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-657::DEBUG::2012-12-09 12:18:50,075::task::588::TaskManager.Task::(_updateState) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::moving from state init -> state preparing
>Thread-657::INFO::2012-12-09 12:18:50,075::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-657::DEBUG::2012-12-09 12:18:50,076::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dbf33cdb-d4a4-445a-9395-862d9ef9eac7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-657::DEBUG::2012-12-09 12:18:50,076::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-657::DEBUG::2012-12-09 12:18:50,077::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-657::DEBUG::2012-12-09 12:18:50,077::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dbf33cdb-d4a4-445a-9395-862d9ef9eac7`::Granted request
>Thread-657::DEBUG::2012-12-09 12:18:50,078::task::817::TaskManager.Task::(resourceAcquired) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-657::DEBUG::2012-12-09 12:18:50,078::task::978::TaskManager.Task::(_decref) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::ref 1 aborting False
>Thread-657::DEBUG::2012-12-09 12:18:50,080::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-657::DEBUG::2012-12-09 12:18:50,080::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-657::DEBUG::2012-12-09 12:18:50,196::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-657::WARNING::2012-12-09 12:18:50,198::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-657::DEBUG::2012-12-09 12:18:50,199::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-657::WARNING::2012-12-09 12:18:50,200::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-657::INFO::2012-12-09 12:18:50,200::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-657::DEBUG::2012-12-09 12:18:50,201::task::1172::TaskManager.Task::(prepare) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-657::DEBUG::2012-12-09 12:18:50,201::task::588::TaskManager.Task::(_updateState) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::moving from state preparing -> state finished
>Thread-657::DEBUG::2012-12-09 12:18:50,201::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-657::DEBUG::2012-12-09 12:18:50,201::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-657::DEBUG::2012-12-09 12:18:50,201::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-657::DEBUG::2012-12-09 12:18:50,202::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-657::DEBUG::2012-12-09 12:18:50,202::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-657::DEBUG::2012-12-09 12:18:50,202::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-657::DEBUG::2012-12-09 12:18:50,202::task::978::TaskManager.Task::(_decref) Task=`6090621b-c8a8-488b-a96a-81bf024bc1e3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:18:50,276::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:50,325::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0242528 s, 42.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:52,332::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:52,385::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277229 s, 36.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:54,392::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:54,443::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274364 s, 37.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:18:56,132::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:18:56,148::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000850305 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:56,450::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:56,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237375 s, 43.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:18:58,505::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:18:58,553::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214738 s, 47.7 MB/s\n'; <rc> = 0
>Thread-662::DEBUG::2012-12-09 12:18:59,542::task::588::TaskManager.Task::(_updateState) Task=`f4bae88d-9f19-413e-bc6a-9ede93f5ff81`::moving from state init -> state preparing
>Thread-662::INFO::2012-12-09 12:18:59,543::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-662::INFO::2012-12-09 12:18:59,543::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166869163513', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-662::DEBUG::2012-12-09 12:18:59,543::task::1172::TaskManager.Task::(prepare) Task=`f4bae88d-9f19-413e-bc6a-9ede93f5ff81`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166869163513', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-662::DEBUG::2012-12-09 12:18:59,543::task::588::TaskManager.Task::(_updateState) Task=`f4bae88d-9f19-413e-bc6a-9ede93f5ff81`::moving from state preparing -> state finished
>Thread-662::DEBUG::2012-12-09 12:18:59,544::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-662::DEBUG::2012-12-09 12:18:59,544::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-662::DEBUG::2012-12-09 12:18:59,544::task::978::TaskManager.Task::(_decref) Task=`f4bae88d-9f19-413e-bc6a-9ede93f5ff81`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:18:59,908::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:18:59,909::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:19:00,019::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:00,020::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:00,020::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:00,020::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:19:00,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:00,606::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223188 s, 45.9 MB/s\n'; <rc> = 0
>Thread-664::DEBUG::2012-12-09 12:19:00,646::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-664::DEBUG::2012-12-09 12:19:00,647::task::588::TaskManager.Task::(_updateState) Task=`7ed94fec-25fb-4ce7-b69a-911654d30a21`::moving from state init -> state preparing
>Thread-664::INFO::2012-12-09 12:19:00,647::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-664::INFO::2012-12-09 12:19:00,648::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-664::DEBUG::2012-12-09 12:19:00,648::task::1172::TaskManager.Task::(prepare) Task=`7ed94fec-25fb-4ce7-b69a-911654d30a21`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-664::DEBUG::2012-12-09 12:19:00,649::task::588::TaskManager.Task::(_updateState) Task=`7ed94fec-25fb-4ce7-b69a-911654d30a21`::moving from state preparing -> state finished
>Thread-664::DEBUG::2012-12-09 12:19:00,649::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-664::DEBUG::2012-12-09 12:19:00,649::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-664::DEBUG::2012-12-09 12:19:00,650::task::978::TaskManager.Task::(_decref) Task=`7ed94fec-25fb-4ce7-b69a-911654d30a21`::ref 0 aborting False
>Thread-665::DEBUG::2012-12-09 12:19:00,665::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-665::DEBUG::2012-12-09 12:19:00,665::task::588::TaskManager.Task::(_updateState) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::moving from state init -> state preparing
>Thread-665::INFO::2012-12-09 12:19:00,666::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-665::DEBUG::2012-12-09 12:19:00,667::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a8b28cee-02d1-473d-b071-9256824ebcb5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-665::DEBUG::2012-12-09 12:19:00,667::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-665::DEBUG::2012-12-09 12:19:00,667::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-665::DEBUG::2012-12-09 12:19:00,667::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a8b28cee-02d1-473d-b071-9256824ebcb5`::Granted request
>Thread-665::DEBUG::2012-12-09 12:19:00,668::task::817::TaskManager.Task::(resourceAcquired) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-665::DEBUG::2012-12-09 12:19:00,668::task::978::TaskManager.Task::(_decref) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::ref 1 aborting False
>Thread-665::DEBUG::2012-12-09 12:19:00,670::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-665::DEBUG::2012-12-09 12:19:00,670::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-665::DEBUG::2012-12-09 12:19:00,777::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-665::WARNING::2012-12-09 12:19:00,779::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-665::DEBUG::2012-12-09 12:19:00,779::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-665::WARNING::2012-12-09 12:19:00,780::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-665::INFO::2012-12-09 12:19:00,781::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-665::DEBUG::2012-12-09 12:19:00,782::task::1172::TaskManager.Task::(prepare) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-665::DEBUG::2012-12-09 12:19:00,782::task::588::TaskManager.Task::(_updateState) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::moving from state preparing -> state finished
>Thread-665::DEBUG::2012-12-09 12:19:00,783::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-665::DEBUG::2012-12-09 12:19:00,783::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-665::DEBUG::2012-12-09 12:19:00,784::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-665::DEBUG::2012-12-09 12:19:00,784::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-665::DEBUG::2012-12-09 12:19:00,785::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-665::DEBUG::2012-12-09 12:19:00,785::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-665::DEBUG::2012-12-09 12:19:00,786::task::978::TaskManager.Task::(_decref) Task=`caf2f0c3-b68c-43ae-ba40-a9b57f66b4e3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:02,613::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:02,664::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267192 s, 38.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:04,671::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:04,719::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225773 s, 45.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:06,152::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:19:06,153::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:19:06,154::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:06,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000880933 s, 2.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:06,168::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000880933 s, 2.3 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:19:06,169::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:19:06,169::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:19:06,170::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:19:06,170::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,170::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,171::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,171::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,171::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,171::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:06,280::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:06,282::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:19:06,282::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:19:06,282::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:19:06,283::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:19:06,283::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:19:06,283::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:06,391::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:06,391::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:06,400::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000622042 s, 6.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:06,726::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:06,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186634 s, 54.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:08,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:08,821::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229955 s, 44.5 MB/s\n'; <rc> = 0
>Thread-670::DEBUG::2012-12-09 12:19:09,853::task::588::TaskManager.Task::(_updateState) Task=`322e6752-46f6-4603-859f-df2e62d9042b`::moving from state init -> state preparing
>Thread-670::INFO::2012-12-09 12:19:09,854::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-670::INFO::2012-12-09 12:19:09,854::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.008868932724', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-670::DEBUG::2012-12-09 12:19:09,855::task::1172::TaskManager.Task::(prepare) Task=`322e6752-46f6-4603-859f-df2e62d9042b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.008868932724', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-670::DEBUG::2012-12-09 12:19:09,855::task::588::TaskManager.Task::(_updateState) Task=`322e6752-46f6-4603-859f-df2e62d9042b`::moving from state preparing -> state finished
>Thread-670::DEBUG::2012-12-09 12:19:09,855::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-670::DEBUG::2012-12-09 12:19:09,856::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-670::DEBUG::2012-12-09 12:19:09,856::task::978::TaskManager.Task::(_decref) Task=`322e6752-46f6-4603-859f-df2e62d9042b`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:19:10,021::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:19:10,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:19:10,023::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:19:10,133::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:10,135::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:10,136::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:10,137::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:19:10,827::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:10,876::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0232795 s, 44.0 MB/s\n'; <rc> = 0
>Thread-672::DEBUG::2012-12-09 12:19:11,405::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-672::DEBUG::2012-12-09 12:19:11,406::task::588::TaskManager.Task::(_updateState) Task=`d703373b-cbfc-4a67-af9e-20abedc8a077`::moving from state init -> state preparing
>Thread-672::INFO::2012-12-09 12:19:11,407::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-672::INFO::2012-12-09 12:19:11,407::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-672::DEBUG::2012-12-09 12:19:11,408::task::1172::TaskManager.Task::(prepare) Task=`d703373b-cbfc-4a67-af9e-20abedc8a077`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-672::DEBUG::2012-12-09 12:19:11,408::task::588::TaskManager.Task::(_updateState) Task=`d703373b-cbfc-4a67-af9e-20abedc8a077`::moving from state preparing -> state finished
>Thread-672::DEBUG::2012-12-09 12:19:11,409::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-672::DEBUG::2012-12-09 12:19:11,409::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-672::DEBUG::2012-12-09 12:19:11,410::task::978::TaskManager.Task::(_decref) Task=`d703373b-cbfc-4a67-af9e-20abedc8a077`::ref 0 aborting False
>Thread-673::DEBUG::2012-12-09 12:19:11,427::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-673::DEBUG::2012-12-09 12:19:11,428::task::588::TaskManager.Task::(_updateState) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::moving from state init -> state preparing
>Thread-673::INFO::2012-12-09 12:19:11,429::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-673::DEBUG::2012-12-09 12:19:11,429::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f53e9818-b1d0-4818-9156-bf60f13c71e6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-673::DEBUG::2012-12-09 12:19:11,430::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-673::DEBUG::2012-12-09 12:19:11,430::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-673::DEBUG::2012-12-09 12:19:11,431::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f53e9818-b1d0-4818-9156-bf60f13c71e6`::Granted request
>Thread-673::DEBUG::2012-12-09 12:19:11,431::task::817::TaskManager.Task::(resourceAcquired) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-673::DEBUG::2012-12-09 12:19:11,432::task::978::TaskManager.Task::(_decref) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::ref 1 aborting False
>Thread-673::DEBUG::2012-12-09 12:19:11,433::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-673::DEBUG::2012-12-09 12:19:11,434::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-673::DEBUG::2012-12-09 12:19:11,546::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-673::WARNING::2012-12-09 12:19:11,547::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-673::DEBUG::2012-12-09 12:19:11,548::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-673::WARNING::2012-12-09 12:19:11,549::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-673::INFO::2012-12-09 12:19:11,549::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-673::DEBUG::2012-12-09 12:19:11,550::task::1172::TaskManager.Task::(prepare) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-673::DEBUG::2012-12-09 12:19:11,550::task::588::TaskManager.Task::(_updateState) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::moving from state preparing -> state finished
>Thread-673::DEBUG::2012-12-09 12:19:11,551::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-673::DEBUG::2012-12-09 12:19:11,551::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-673::DEBUG::2012-12-09 12:19:11,551::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-673::DEBUG::2012-12-09 12:19:11,551::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-673::DEBUG::2012-12-09 12:19:11,551::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-673::DEBUG::2012-12-09 12:19:11,552::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-673::DEBUG::2012-12-09 12:19:11,552::task::978::TaskManager.Task::(_decref) Task=`be5b7b86-0001-4031-9eab-189bbeb9571a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:12,883::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:12,933::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249431 s, 41.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:14,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:14,986::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02237 s, 45.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:16,404::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:16,419::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000929699 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:16,993::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:17,044::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263828 s, 38.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:19,053::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:19,092::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165943 s, 61.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:19:20,138::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:19:20,140::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-678::DEBUG::2012-12-09 12:19:20,167::task::588::TaskManager.Task::(_updateState) Task=`f9b28880-65fc-48ec-8787-8fb90d4dd169`::moving from state init -> state preparing
>Thread-678::INFO::2012-12-09 12:19:20,168::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-678::INFO::2012-12-09 12:19:20,168::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164740085602', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-678::DEBUG::2012-12-09 12:19:20,168::task::1172::TaskManager.Task::(prepare) Task=`f9b28880-65fc-48ec-8787-8fb90d4dd169`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164740085602', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-678::DEBUG::2012-12-09 12:19:20,168::task::588::TaskManager.Task::(_updateState) Task=`f9b28880-65fc-48ec-8787-8fb90d4dd169`::moving from state preparing -> state finished
>Thread-678::DEBUG::2012-12-09 12:19:20,168::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-678::DEBUG::2012-12-09 12:19:20,169::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-678::DEBUG::2012-12-09 12:19:20,169::task::978::TaskManager.Task::(_decref) Task=`f9b28880-65fc-48ec-8787-8fb90d4dd169`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:19:20,241::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:20,242::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:20,243::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:20,243::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:19:21,099::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:21,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181321 s, 56.5 MB/s\n'; <rc> = 0
>Thread-680::DEBUG::2012-12-09 12:19:21,741::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-680::DEBUG::2012-12-09 12:19:21,742::task::588::TaskManager.Task::(_updateState) Task=`cc535ef5-39b2-44f5-8c05-6dcdf78d9493`::moving from state init -> state preparing
>Thread-680::INFO::2012-12-09 12:19:21,743::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-680::INFO::2012-12-09 12:19:21,744::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-680::DEBUG::2012-12-09 12:19:21,744::task::1172::TaskManager.Task::(prepare) Task=`cc535ef5-39b2-44f5-8c05-6dcdf78d9493`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-680::DEBUG::2012-12-09 12:19:21,745::task::588::TaskManager.Task::(_updateState) Task=`cc535ef5-39b2-44f5-8c05-6dcdf78d9493`::moving from state preparing -> state finished
>Thread-680::DEBUG::2012-12-09 12:19:21,746::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-680::DEBUG::2012-12-09 12:19:21,746::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-680::DEBUG::2012-12-09 12:19:21,747::task::978::TaskManager.Task::(_decref) Task=`cc535ef5-39b2-44f5-8c05-6dcdf78d9493`::ref 0 aborting False
>Thread-681::DEBUG::2012-12-09 12:19:21,762::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-681::DEBUG::2012-12-09 12:19:21,763::task::588::TaskManager.Task::(_updateState) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::moving from state init -> state preparing
>Thread-681::INFO::2012-12-09 12:19:21,763::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-681::DEBUG::2012-12-09 12:19:21,764::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`75b9478b-b94c-4747-adc4-5b1fe42160f7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-681::DEBUG::2012-12-09 12:19:21,765::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-681::DEBUG::2012-12-09 12:19:21,765::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-681::DEBUG::2012-12-09 12:19:21,766::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`75b9478b-b94c-4747-adc4-5b1fe42160f7`::Granted request
>Thread-681::DEBUG::2012-12-09 12:19:21,766::task::817::TaskManager.Task::(resourceAcquired) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-681::DEBUG::2012-12-09 12:19:21,767::task::978::TaskManager.Task::(_decref) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::ref 1 aborting False
>Thread-681::DEBUG::2012-12-09 12:19:21,768::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-681::DEBUG::2012-12-09 12:19:21,769::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-681::DEBUG::2012-12-09 12:19:21,875::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-681::WARNING::2012-12-09 12:19:21,876::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-681::DEBUG::2012-12-09 12:19:21,876::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-681::WARNING::2012-12-09 12:19:21,877::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-681::INFO::2012-12-09 12:19:21,877::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-681::DEBUG::2012-12-09 12:19:21,877::task::1172::TaskManager.Task::(prepare) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-681::DEBUG::2012-12-09 12:19:21,878::task::588::TaskManager.Task::(_updateState) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::moving from state preparing -> state finished
>Thread-681::DEBUG::2012-12-09 12:19:21,878::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-681::DEBUG::2012-12-09 12:19:21,878::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-681::DEBUG::2012-12-09 12:19:21,878::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-681::DEBUG::2012-12-09 12:19:21,878::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-681::DEBUG::2012-12-09 12:19:21,879::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-681::DEBUG::2012-12-09 12:19:21,879::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-681::DEBUG::2012-12-09 12:19:21,879::task::978::TaskManager.Task::(_decref) Task=`97fbc533-0d89-40d9-9cd2-e970bc1d7fdb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:23,150::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:23,203::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278529 s, 36.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:25,210::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:25,258::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224331 s, 45.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:26,423::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:26,437::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000768492 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:27,264::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:27,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255433 s, 40.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:29,322::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:29,368::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225068 s, 45.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:19:30,244::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:19:30,245::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:19:30,360::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:30,361::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:30,362::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:30,362::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-686::DEBUG::2012-12-09 12:19:30,388::task::588::TaskManager.Task::(_updateState) Task=`266328b3-21d5-47b6-a4e8-4f0752583fa4`::moving from state init -> state preparing
>Thread-686::INFO::2012-12-09 12:19:30,389::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-686::INFO::2012-12-09 12:19:30,389::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146329402924', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-686::DEBUG::2012-12-09 12:19:30,390::task::1172::TaskManager.Task::(prepare) Task=`266328b3-21d5-47b6-a4e8-4f0752583fa4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146329402924', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-686::DEBUG::2012-12-09 12:19:30,390::task::588::TaskManager.Task::(_updateState) Task=`266328b3-21d5-47b6-a4e8-4f0752583fa4`::moving from state preparing -> state finished
>Thread-686::DEBUG::2012-12-09 12:19:30,391::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-686::DEBUG::2012-12-09 12:19:30,391::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-686::DEBUG::2012-12-09 12:19:30,391::task::978::TaskManager.Task::(_decref) Task=`266328b3-21d5-47b6-a4e8-4f0752583fa4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:31,375::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:31,425::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254275 s, 40.3 MB/s\n'; <rc> = 0
>Thread-688::DEBUG::2012-12-09 12:19:32,037::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-688::DEBUG::2012-12-09 12:19:32,038::task::588::TaskManager.Task::(_updateState) Task=`df3c3dae-5c1f-4973-a93a-a5742853c934`::moving from state init -> state preparing
>Thread-688::INFO::2012-12-09 12:19:32,039::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-688::INFO::2012-12-09 12:19:32,039::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-688::DEBUG::2012-12-09 12:19:32,040::task::1172::TaskManager.Task::(prepare) Task=`df3c3dae-5c1f-4973-a93a-a5742853c934`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-688::DEBUG::2012-12-09 12:19:32,040::task::588::TaskManager.Task::(_updateState) Task=`df3c3dae-5c1f-4973-a93a-a5742853c934`::moving from state preparing -> state finished
>Thread-688::DEBUG::2012-12-09 12:19:32,040::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-688::DEBUG::2012-12-09 12:19:32,040::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-688::DEBUG::2012-12-09 12:19:32,041::task::978::TaskManager.Task::(_decref) Task=`df3c3dae-5c1f-4973-a93a-a5742853c934`::ref 0 aborting False
>Thread-689::DEBUG::2012-12-09 12:19:32,062::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-689::DEBUG::2012-12-09 12:19:32,062::task::588::TaskManager.Task::(_updateState) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::moving from state init -> state preparing
>Thread-689::INFO::2012-12-09 12:19:32,063::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-689::DEBUG::2012-12-09 12:19:32,064::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a19a6b0-ecab-421b-a0cf-482b74a14cd9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-689::DEBUG::2012-12-09 12:19:32,064::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-689::DEBUG::2012-12-09 12:19:32,065::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-689::DEBUG::2012-12-09 12:19:32,065::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a19a6b0-ecab-421b-a0cf-482b74a14cd9`::Granted request
>Thread-689::DEBUG::2012-12-09 12:19:32,066::task::817::TaskManager.Task::(resourceAcquired) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-689::DEBUG::2012-12-09 12:19:32,066::task::978::TaskManager.Task::(_decref) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::ref 1 aborting False
>Thread-689::DEBUG::2012-12-09 12:19:32,068::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-689::DEBUG::2012-12-09 12:19:32,069::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-689::DEBUG::2012-12-09 12:19:32,180::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-689::WARNING::2012-12-09 12:19:32,182::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-689::DEBUG::2012-12-09 12:19:32,183::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-689::WARNING::2012-12-09 12:19:32,184::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-689::INFO::2012-12-09 12:19:32,184::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-689::DEBUG::2012-12-09 12:19:32,185::task::1172::TaskManager.Task::(prepare) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-689::DEBUG::2012-12-09 12:19:32,185::task::588::TaskManager.Task::(_updateState) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::moving from state preparing -> state finished
>Thread-689::DEBUG::2012-12-09 12:19:32,186::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-689::DEBUG::2012-12-09 12:19:32,186::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-689::DEBUG::2012-12-09 12:19:32,186::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-689::DEBUG::2012-12-09 12:19:32,187::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-689::DEBUG::2012-12-09 12:19:32,187::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-689::DEBUG::2012-12-09 12:19:32,188::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-689::DEBUG::2012-12-09 12:19:32,188::task::978::TaskManager.Task::(_decref) Task=`7df34f15-93a5-42bd-a718-61fb142822d0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:33,431::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:33,477::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210478 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:35,483::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:35,534::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027168 s, 37.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:36,440::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:36,456::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000917924 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:37,541::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:37,593::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276881 s, 37.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:39,600::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:39,647::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224715 s, 45.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:19:40,363::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:19:40,364::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:19:40,475::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:40,477::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:40,478::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:40,479::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-694::DEBUG::2012-12-09 12:19:40,671::task::588::TaskManager.Task::(_updateState) Task=`95f71da9-2547-4c47-b85c-0a937e2a3787`::moving from state init -> state preparing
>Thread-694::INFO::2012-12-09 12:19:40,671::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-694::INFO::2012-12-09 12:19:40,672::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168540477753', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-694::DEBUG::2012-12-09 12:19:40,672::task::1172::TaskManager.Task::(prepare) Task=`95f71da9-2547-4c47-b85c-0a937e2a3787`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168540477753', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-694::DEBUG::2012-12-09 12:19:40,673::task::588::TaskManager.Task::(_updateState) Task=`95f71da9-2547-4c47-b85c-0a937e2a3787`::moving from state preparing -> state finished
>Thread-694::DEBUG::2012-12-09 12:19:40,673::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-694::DEBUG::2012-12-09 12:19:40,673::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-694::DEBUG::2012-12-09 12:19:40,674::task::978::TaskManager.Task::(_decref) Task=`95f71da9-2547-4c47-b85c-0a937e2a3787`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:41,653::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:41,704::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255899 s, 40.0 MB/s\n'; <rc> = 0
>Thread-696::DEBUG::2012-12-09 12:19:42,352::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-696::DEBUG::2012-12-09 12:19:42,352::task::588::TaskManager.Task::(_updateState) Task=`b3f7f5e6-5ea9-4245-b543-8f30191e38a6`::moving from state init -> state preparing
>Thread-696::INFO::2012-12-09 12:19:42,352::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-696::INFO::2012-12-09 12:19:42,353::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-696::DEBUG::2012-12-09 12:19:42,353::task::1172::TaskManager.Task::(prepare) Task=`b3f7f5e6-5ea9-4245-b543-8f30191e38a6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-696::DEBUG::2012-12-09 12:19:42,353::task::588::TaskManager.Task::(_updateState) Task=`b3f7f5e6-5ea9-4245-b543-8f30191e38a6`::moving from state preparing -> state finished
>Thread-696::DEBUG::2012-12-09 12:19:42,353::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-696::DEBUG::2012-12-09 12:19:42,353::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-696::DEBUG::2012-12-09 12:19:42,354::task::978::TaskManager.Task::(_decref) Task=`b3f7f5e6-5ea9-4245-b543-8f30191e38a6`::ref 0 aborting False
>Thread-697::DEBUG::2012-12-09 12:19:42,367::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-697::DEBUG::2012-12-09 12:19:42,367::task::588::TaskManager.Task::(_updateState) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::moving from state init -> state preparing
>Thread-697::INFO::2012-12-09 12:19:42,367::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-697::DEBUG::2012-12-09 12:19:42,368::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1754f901-e7d0-4211-99ca-b6a899973b87`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-697::DEBUG::2012-12-09 12:19:42,368::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-697::DEBUG::2012-12-09 12:19:42,368::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-697::DEBUG::2012-12-09 12:19:42,368::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1754f901-e7d0-4211-99ca-b6a899973b87`::Granted request
>Thread-697::DEBUG::2012-12-09 12:19:42,369::task::817::TaskManager.Task::(resourceAcquired) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-697::DEBUG::2012-12-09 12:19:42,369::task::978::TaskManager.Task::(_decref) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::ref 1 aborting False
>Thread-697::DEBUG::2012-12-09 12:19:42,370::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-697::DEBUG::2012-12-09 12:19:42,370::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-697::DEBUG::2012-12-09 12:19:42,466::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-697::WARNING::2012-12-09 12:19:42,468::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-697::DEBUG::2012-12-09 12:19:42,468::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-697::WARNING::2012-12-09 12:19:42,468::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-697::INFO::2012-12-09 12:19:42,469::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-697::DEBUG::2012-12-09 12:19:42,469::task::1172::TaskManager.Task::(prepare) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-697::DEBUG::2012-12-09 12:19:42,469::task::588::TaskManager.Task::(_updateState) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::moving from state preparing -> state finished
>Thread-697::DEBUG::2012-12-09 12:19:42,470::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-697::DEBUG::2012-12-09 12:19:42,470::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-697::DEBUG::2012-12-09 12:19:42,470::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-697::DEBUG::2012-12-09 12:19:42,471::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-697::DEBUG::2012-12-09 12:19:42,471::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-697::DEBUG::2012-12-09 12:19:42,471::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-697::DEBUG::2012-12-09 12:19:42,472::task::978::TaskManager.Task::(_decref) Task=`e0fe361d-46a3-44d7-bb91-089d91bf8ed9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:43,711::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:43,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217882 s, 47.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:45,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:45,827::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277901 s, 36.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:46,462::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:46,477::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000933646 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:47,832::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:47,884::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272889 s, 37.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:49,891::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:49,943::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277723 s, 36.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:19:50,480::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:19:50,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:19:50,601::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:19:50,602::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:19:50,602::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:19:50,603::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-702::DEBUG::2012-12-09 12:19:50,936::task::588::TaskManager.Task::(_updateState) Task=`71e27e9f-b1d9-415e-ac2a-ca34ec52e0b9`::moving from state init -> state preparing
>Thread-702::INFO::2012-12-09 12:19:50,936::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-702::INFO::2012-12-09 12:19:50,937::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167140960693', 'lastCheck': '4.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-702::DEBUG::2012-12-09 12:19:50,937::task::1172::TaskManager.Task::(prepare) Task=`71e27e9f-b1d9-415e-ac2a-ca34ec52e0b9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167140960693', 'lastCheck': '4.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-702::DEBUG::2012-12-09 12:19:50,938::task::588::TaskManager.Task::(_updateState) Task=`71e27e9f-b1d9-415e-ac2a-ca34ec52e0b9`::moving from state preparing -> state finished
>Thread-702::DEBUG::2012-12-09 12:19:50,938::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-702::DEBUG::2012-12-09 12:19:50,938::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-702::DEBUG::2012-12-09 12:19:50,939::task::978::TaskManager.Task::(_decref) Task=`71e27e9f-b1d9-415e-ac2a-ca34ec52e0b9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:51,950::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:51,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217103 s, 47.2 MB/s\n'; <rc> = 0
>Thread-704::DEBUG::2012-12-09 12:19:52,631::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-704::DEBUG::2012-12-09 12:19:52,632::task::588::TaskManager.Task::(_updateState) Task=`f1e9882c-54a8-410c-929e-75a08f694c82`::moving from state init -> state preparing
>Thread-704::INFO::2012-12-09 12:19:52,633::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-704::INFO::2012-12-09 12:19:52,634::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-704::DEBUG::2012-12-09 12:19:52,634::task::1172::TaskManager.Task::(prepare) Task=`f1e9882c-54a8-410c-929e-75a08f694c82`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-704::DEBUG::2012-12-09 12:19:52,635::task::588::TaskManager.Task::(_updateState) Task=`f1e9882c-54a8-410c-929e-75a08f694c82`::moving from state preparing -> state finished
>Thread-704::DEBUG::2012-12-09 12:19:52,635::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-704::DEBUG::2012-12-09 12:19:52,635::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-704::DEBUG::2012-12-09 12:19:52,636::task::978::TaskManager.Task::(_decref) Task=`f1e9882c-54a8-410c-929e-75a08f694c82`::ref 0 aborting False
>Thread-705::DEBUG::2012-12-09 12:19:52,655::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-705::DEBUG::2012-12-09 12:19:52,656::task::588::TaskManager.Task::(_updateState) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::moving from state init -> state preparing
>Thread-705::INFO::2012-12-09 12:19:52,657::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-705::DEBUG::2012-12-09 12:19:52,658::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`53144c7c-3f77-4e51-b91e-1f10f0dcef41`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-705::DEBUG::2012-12-09 12:19:52,658::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-705::DEBUG::2012-12-09 12:19:52,658::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-705::DEBUG::2012-12-09 12:19:52,659::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`53144c7c-3f77-4e51-b91e-1f10f0dcef41`::Granted request
>Thread-705::DEBUG::2012-12-09 12:19:52,660::task::817::TaskManager.Task::(resourceAcquired) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-705::DEBUG::2012-12-09 12:19:52,660::task::978::TaskManager.Task::(_decref) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::ref 1 aborting False
>Thread-705::DEBUG::2012-12-09 12:19:52,661::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-705::DEBUG::2012-12-09 12:19:52,661::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-705::DEBUG::2012-12-09 12:19:52,773::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-705::WARNING::2012-12-09 12:19:52,775::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-705::DEBUG::2012-12-09 12:19:52,776::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-705::WARNING::2012-12-09 12:19:52,777::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-705::INFO::2012-12-09 12:19:52,778::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-705::DEBUG::2012-12-09 12:19:52,778::task::1172::TaskManager.Task::(prepare) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-705::DEBUG::2012-12-09 12:19:52,779::task::588::TaskManager.Task::(_updateState) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::moving from state preparing -> state finished
>Thread-705::DEBUG::2012-12-09 12:19:52,779::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-705::DEBUG::2012-12-09 12:19:52,780::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-705::DEBUG::2012-12-09 12:19:52,780::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-705::DEBUG::2012-12-09 12:19:52,781::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-705::DEBUG::2012-12-09 12:19:52,781::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-705::DEBUG::2012-12-09 12:19:52,782::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-705::DEBUG::2012-12-09 12:19:52,782::task::978::TaskManager.Task::(_decref) Task=`2b1ad82f-8927-4e7b-9a90-3b5dc33605d5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:19:54,003::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:54,048::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207122 s, 49.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:56,054::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:56,100::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020004 s, 51.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:19:56,482::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:19:56,497::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000795094 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:19:58,111::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:19:58,164::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275249 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:00,170::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:00,225::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0291674 s, 35.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:00,603::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:00,604::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:00,706::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:00,707::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:00,707::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:00,708::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-710::DEBUG::2012-12-09 12:20:01,224::task::588::TaskManager.Task::(_updateState) Task=`9cad3f19-7b52-4525-b459-55ebb5d3c279`::moving from state init -> state preparing
>Thread-710::INFO::2012-12-09 12:20:01,225::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-710::INFO::2012-12-09 12:20:01,225::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160369873047', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-710::DEBUG::2012-12-09 12:20:01,225::task::1172::TaskManager.Task::(prepare) Task=`9cad3f19-7b52-4525-b459-55ebb5d3c279`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160369873047', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-710::DEBUG::2012-12-09 12:20:01,225::task::588::TaskManager.Task::(_updateState) Task=`9cad3f19-7b52-4525-b459-55ebb5d3c279`::moving from state preparing -> state finished
>Thread-710::DEBUG::2012-12-09 12:20:01,225::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-710::DEBUG::2012-12-09 12:20:01,226::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-710::DEBUG::2012-12-09 12:20:01,226::task::978::TaskManager.Task::(_decref) Task=`9cad3f19-7b52-4525-b459-55ebb5d3c279`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:02,231::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:02,284::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272426 s, 37.6 MB/s\n'; <rc> = 0
>Thread-712::DEBUG::2012-12-09 12:20:02,931::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-712::DEBUG::2012-12-09 12:20:02,932::task::588::TaskManager.Task::(_updateState) Task=`2bec35e1-5fe1-4c9e-8f0f-860a92e0fcc1`::moving from state init -> state preparing
>Thread-712::INFO::2012-12-09 12:20:02,933::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-712::INFO::2012-12-09 12:20:02,933::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-712::DEBUG::2012-12-09 12:20:02,934::task::1172::TaskManager.Task::(prepare) Task=`2bec35e1-5fe1-4c9e-8f0f-860a92e0fcc1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-712::DEBUG::2012-12-09 12:20:02,934::task::588::TaskManager.Task::(_updateState) Task=`2bec35e1-5fe1-4c9e-8f0f-860a92e0fcc1`::moving from state preparing -> state finished
>Thread-712::DEBUG::2012-12-09 12:20:02,935::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-712::DEBUG::2012-12-09 12:20:02,935::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-712::DEBUG::2012-12-09 12:20:02,935::task::978::TaskManager.Task::(_decref) Task=`2bec35e1-5fe1-4c9e-8f0f-860a92e0fcc1`::ref 0 aborting False
>Thread-713::DEBUG::2012-12-09 12:20:02,955::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-713::DEBUG::2012-12-09 12:20:02,956::task::588::TaskManager.Task::(_updateState) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::moving from state init -> state preparing
>Thread-713::INFO::2012-12-09 12:20:02,956::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-713::DEBUG::2012-12-09 12:20:02,957::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`98996bd7-afc0-4898-84be-1735e3bd1100`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-713::DEBUG::2012-12-09 12:20:02,957::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-713::DEBUG::2012-12-09 12:20:02,958::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-713::DEBUG::2012-12-09 12:20:02,958::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`98996bd7-afc0-4898-84be-1735e3bd1100`::Granted request
>Thread-713::DEBUG::2012-12-09 12:20:02,959::task::817::TaskManager.Task::(resourceAcquired) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-713::DEBUG::2012-12-09 12:20:02,959::task::978::TaskManager.Task::(_decref) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::ref 1 aborting False
>Thread-713::DEBUG::2012-12-09 12:20:02,961::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-713::DEBUG::2012-12-09 12:20:02,961::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-713::DEBUG::2012-12-09 12:20:03,070::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-713::WARNING::2012-12-09 12:20:03,071::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-713::DEBUG::2012-12-09 12:20:03,072::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-713::WARNING::2012-12-09 12:20:03,072::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-713::INFO::2012-12-09 12:20:03,073::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-713::DEBUG::2012-12-09 12:20:03,073::task::1172::TaskManager.Task::(prepare) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-713::DEBUG::2012-12-09 12:20:03,073::task::588::TaskManager.Task::(_updateState) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::moving from state preparing -> state finished
>Thread-713::DEBUG::2012-12-09 12:20:03,073::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-713::DEBUG::2012-12-09 12:20:03,073::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-713::DEBUG::2012-12-09 12:20:03,074::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-713::DEBUG::2012-12-09 12:20:03,074::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-713::DEBUG::2012-12-09 12:20:03,074::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-713::DEBUG::2012-12-09 12:20:03,074::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-713::DEBUG::2012-12-09 12:20:03,074::task::978::TaskManager.Task::(_decref) Task=`63e4ce1f-f17e-47cc-9ec5-fa177979d05e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:04,290::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:04,342::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215264 s, 47.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:06,349::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:06,398::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239927 s, 42.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:06,502::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:06,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000608638 s, 6.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:08,404::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:08,455::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254699 s, 40.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:10,461::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:10,514::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028558 s, 35.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:10,708::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:10,710::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:10,820::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:10,821::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:10,821::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:10,822::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-718::DEBUG::2012-12-09 12:20:11,480::task::588::TaskManager.Task::(_updateState) Task=`55f4d236-7ad9-45f8-8d08-132e8f3003d0`::moving from state init -> state preparing
>Thread-718::INFO::2012-12-09 12:20:11,481::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-718::INFO::2012-12-09 12:20:11,481::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140209197998', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-718::DEBUG::2012-12-09 12:20:11,481::task::1172::TaskManager.Task::(prepare) Task=`55f4d236-7ad9-45f8-8d08-132e8f3003d0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140209197998', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-718::DEBUG::2012-12-09 12:20:11,481::task::588::TaskManager.Task::(_updateState) Task=`55f4d236-7ad9-45f8-8d08-132e8f3003d0`::moving from state preparing -> state finished
>Thread-718::DEBUG::2012-12-09 12:20:11,482::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-718::DEBUG::2012-12-09 12:20:11,482::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-718::DEBUG::2012-12-09 12:20:11,482::task::978::TaskManager.Task::(_decref) Task=`55f4d236-7ad9-45f8-8d08-132e8f3003d0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:12,520::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:12,567::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221977 s, 46.1 MB/s\n'; <rc> = 0
>Thread-720::DEBUG::2012-12-09 12:20:13,223::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-720::DEBUG::2012-12-09 12:20:13,224::task::588::TaskManager.Task::(_updateState) Task=`9ecb0676-5b40-4ffe-8d75-5aaa46ab82c9`::moving from state init -> state preparing
>Thread-720::INFO::2012-12-09 12:20:13,224::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-720::INFO::2012-12-09 12:20:13,224::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-720::DEBUG::2012-12-09 12:20:13,224::task::1172::TaskManager.Task::(prepare) Task=`9ecb0676-5b40-4ffe-8d75-5aaa46ab82c9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-720::DEBUG::2012-12-09 12:20:13,225::task::588::TaskManager.Task::(_updateState) Task=`9ecb0676-5b40-4ffe-8d75-5aaa46ab82c9`::moving from state preparing -> state finished
>Thread-720::DEBUG::2012-12-09 12:20:13,225::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-720::DEBUG::2012-12-09 12:20:13,225::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-720::DEBUG::2012-12-09 12:20:13,225::task::978::TaskManager.Task::(_decref) Task=`9ecb0676-5b40-4ffe-8d75-5aaa46ab82c9`::ref 0 aborting False
>Thread-721::DEBUG::2012-12-09 12:20:13,238::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-721::DEBUG::2012-12-09 12:20:13,239::task::588::TaskManager.Task::(_updateState) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::moving from state init -> state preparing
>Thread-721::INFO::2012-12-09 12:20:13,239::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-721::DEBUG::2012-12-09 12:20:13,239::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`14e5e7db-4575-432e-8d9e-aeb0f4b0fd1a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-721::DEBUG::2012-12-09 12:20:13,240::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-721::DEBUG::2012-12-09 12:20:13,240::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-721::DEBUG::2012-12-09 12:20:13,241::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`14e5e7db-4575-432e-8d9e-aeb0f4b0fd1a`::Granted request
>Thread-721::DEBUG::2012-12-09 12:20:13,241::task::817::TaskManager.Task::(resourceAcquired) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-721::DEBUG::2012-12-09 12:20:13,241::task::978::TaskManager.Task::(_decref) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::ref 1 aborting False
>Thread-721::DEBUG::2012-12-09 12:20:13,242::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-721::DEBUG::2012-12-09 12:20:13,242::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-721::DEBUG::2012-12-09 12:20:13,347::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-721::WARNING::2012-12-09 12:20:13,348::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-721::DEBUG::2012-12-09 12:20:13,348::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-721::WARNING::2012-12-09 12:20:13,349::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-721::INFO::2012-12-09 12:20:13,349::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-721::DEBUG::2012-12-09 12:20:13,349::task::1172::TaskManager.Task::(prepare) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-721::DEBUG::2012-12-09 12:20:13,350::task::588::TaskManager.Task::(_updateState) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::moving from state preparing -> state finished
>Thread-721::DEBUG::2012-12-09 12:20:13,350::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-721::DEBUG::2012-12-09 12:20:13,350::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-721::DEBUG::2012-12-09 12:20:13,350::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-721::DEBUG::2012-12-09 12:20:13,350::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-721::DEBUG::2012-12-09 12:20:13,351::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-721::DEBUG::2012-12-09 12:20:13,351::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-721::DEBUG::2012-12-09 12:20:13,351::task::978::TaskManager.Task::(_decref) Task=`d855849c-912d-49ac-95e2-aff2f5ca742d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:14,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:14,627::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255303 s, 40.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:16,519::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:16,534::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000849209 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:16,634::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:16,685::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264952 s, 38.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:18,691::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:18,744::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278046 s, 36.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:20,750::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:20,798::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223557 s, 45.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:20,823::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:20,824::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:20,933::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:20,935::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:20,936::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:20,937::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-726::DEBUG::2012-12-09 12:20:21,732::task::588::TaskManager.Task::(_updateState) Task=`5cd9ffc3-d9ef-456c-94c1-dc94cb28d344`::moving from state init -> state preparing
>Thread-726::INFO::2012-12-09 12:20:21,732::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-726::INFO::2012-12-09 12:20:21,733::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.015969991684', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-726::DEBUG::2012-12-09 12:20:21,733::task::1172::TaskManager.Task::(prepare) Task=`5cd9ffc3-d9ef-456c-94c1-dc94cb28d344`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.015969991684', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-726::DEBUG::2012-12-09 12:20:21,734::task::588::TaskManager.Task::(_updateState) Task=`5cd9ffc3-d9ef-456c-94c1-dc94cb28d344`::moving from state preparing -> state finished
>Thread-726::DEBUG::2012-12-09 12:20:21,734::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-726::DEBUG::2012-12-09 12:20:21,735::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-726::DEBUG::2012-12-09 12:20:21,735::task::978::TaskManager.Task::(_decref) Task=`5cd9ffc3-d9ef-456c-94c1-dc94cb28d344`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:22,805::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:22,856::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250898 s, 40.8 MB/s\n'; <rc> = 0
>Thread-728::DEBUG::2012-12-09 12:20:23,462::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-728::DEBUG::2012-12-09 12:20:23,462::task::588::TaskManager.Task::(_updateState) Task=`d3357f2c-a870-43b8-8ec0-5a6f3ae2f399`::moving from state init -> state preparing
>Thread-728::INFO::2012-12-09 12:20:23,463::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-728::INFO::2012-12-09 12:20:23,463::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-728::DEBUG::2012-12-09 12:20:23,464::task::1172::TaskManager.Task::(prepare) Task=`d3357f2c-a870-43b8-8ec0-5a6f3ae2f399`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-728::DEBUG::2012-12-09 12:20:23,464::task::588::TaskManager.Task::(_updateState) Task=`d3357f2c-a870-43b8-8ec0-5a6f3ae2f399`::moving from state preparing -> state finished
>Thread-728::DEBUG::2012-12-09 12:20:23,464::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-728::DEBUG::2012-12-09 12:20:23,465::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-728::DEBUG::2012-12-09 12:20:23,465::task::978::TaskManager.Task::(_decref) Task=`d3357f2c-a870-43b8-8ec0-5a6f3ae2f399`::ref 0 aborting False
>Thread-729::DEBUG::2012-12-09 12:20:23,484::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-729::DEBUG::2012-12-09 12:20:23,485::task::588::TaskManager.Task::(_updateState) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::moving from state init -> state preparing
>Thread-729::INFO::2012-12-09 12:20:23,485::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-729::DEBUG::2012-12-09 12:20:23,486::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c8628afb-a67d-4a18-91b0-cdb5cebabf41`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-729::DEBUG::2012-12-09 12:20:23,486::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-729::DEBUG::2012-12-09 12:20:23,487::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-729::DEBUG::2012-12-09 12:20:23,487::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c8628afb-a67d-4a18-91b0-cdb5cebabf41`::Granted request
>Thread-729::DEBUG::2012-12-09 12:20:23,488::task::817::TaskManager.Task::(resourceAcquired) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-729::DEBUG::2012-12-09 12:20:23,488::task::978::TaskManager.Task::(_decref) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::ref 1 aborting False
>Thread-729::DEBUG::2012-12-09 12:20:23,490::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-729::DEBUG::2012-12-09 12:20:23,491::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-729::DEBUG::2012-12-09 12:20:23,599::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-729::WARNING::2012-12-09 12:20:23,601::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-729::DEBUG::2012-12-09 12:20:23,602::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-729::WARNING::2012-12-09 12:20:23,603::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-729::INFO::2012-12-09 12:20:23,604::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-729::DEBUG::2012-12-09 12:20:23,604::task::1172::TaskManager.Task::(prepare) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-729::DEBUG::2012-12-09 12:20:23,605::task::588::TaskManager.Task::(_updateState) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::moving from state preparing -> state finished
>Thread-729::DEBUG::2012-12-09 12:20:23,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-729::DEBUG::2012-12-09 12:20:23,606::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-729::DEBUG::2012-12-09 12:20:23,606::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-729::DEBUG::2012-12-09 12:20:23,607::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-729::DEBUG::2012-12-09 12:20:23,607::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-729::DEBUG::2012-12-09 12:20:23,608::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-729::DEBUG::2012-12-09 12:20:23,608::task::978::TaskManager.Task::(_decref) Task=`48c3a3d9-a3ec-43ce-9fdc-239f340751a8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:24,862::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:24,914::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275261 s, 37.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:26,540::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:26,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000869131 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:26,920::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:26,967::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220527 s, 46.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:28,973::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:29,025::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273046 s, 37.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:30,938::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:30,939::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:31,031::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:31,053::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:31,056::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:31,057::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:31,058::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:20:31,085::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029225 s, 35.0 MB/s\n'; <rc> = 0
>Thread-734::DEBUG::2012-12-09 12:20:32,004::task::588::TaskManager.Task::(_updateState) Task=`11a01022-2519-4499-8058-c8485b6cfb49`::moving from state init -> state preparing
>Thread-734::INFO::2012-12-09 12:20:32,005::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-734::INFO::2012-12-09 12:20:32,005::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164129734039', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-734::DEBUG::2012-12-09 12:20:32,006::task::1172::TaskManager.Task::(prepare) Task=`11a01022-2519-4499-8058-c8485b6cfb49`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164129734039', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-734::DEBUG::2012-12-09 12:20:32,006::task::588::TaskManager.Task::(_updateState) Task=`11a01022-2519-4499-8058-c8485b6cfb49`::moving from state preparing -> state finished
>Thread-734::DEBUG::2012-12-09 12:20:32,007::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-734::DEBUG::2012-12-09 12:20:32,007::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-734::DEBUG::2012-12-09 12:20:32,008::task::978::TaskManager.Task::(_decref) Task=`11a01022-2519-4499-8058-c8485b6cfb49`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:33,092::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:33,138::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226618 s, 45.2 MB/s\n'; <rc> = 0
>Thread-736::DEBUG::2012-12-09 12:20:33,803::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-736::DEBUG::2012-12-09 12:20:33,804::task::588::TaskManager.Task::(_updateState) Task=`131c75cf-55c2-4b55-b471-5e3b5966118c`::moving from state init -> state preparing
>Thread-736::INFO::2012-12-09 12:20:33,804::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-736::INFO::2012-12-09 12:20:33,805::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-736::DEBUG::2012-12-09 12:20:33,805::task::1172::TaskManager.Task::(prepare) Task=`131c75cf-55c2-4b55-b471-5e3b5966118c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-736::DEBUG::2012-12-09 12:20:33,806::task::588::TaskManager.Task::(_updateState) Task=`131c75cf-55c2-4b55-b471-5e3b5966118c`::moving from state preparing -> state finished
>Thread-736::DEBUG::2012-12-09 12:20:33,806::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-736::DEBUG::2012-12-09 12:20:33,806::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-736::DEBUG::2012-12-09 12:20:33,807::task::978::TaskManager.Task::(_decref) Task=`131c75cf-55c2-4b55-b471-5e3b5966118c`::ref 0 aborting False
>Thread-737::DEBUG::2012-12-09 12:20:33,833::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-737::DEBUG::2012-12-09 12:20:33,833::task::588::TaskManager.Task::(_updateState) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::moving from state init -> state preparing
>Thread-737::INFO::2012-12-09 12:20:33,834::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-737::DEBUG::2012-12-09 12:20:33,835::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79425979-70c6-49c2-ad98-4a72633281f6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-737::DEBUG::2012-12-09 12:20:33,835::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-737::DEBUG::2012-12-09 12:20:33,836::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-737::DEBUG::2012-12-09 12:20:33,836::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79425979-70c6-49c2-ad98-4a72633281f6`::Granted request
>Thread-737::DEBUG::2012-12-09 12:20:33,837::task::817::TaskManager.Task::(resourceAcquired) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-737::DEBUG::2012-12-09 12:20:33,838::task::978::TaskManager.Task::(_decref) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::ref 1 aborting False
>Thread-737::DEBUG::2012-12-09 12:20:33,840::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-737::DEBUG::2012-12-09 12:20:33,841::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-737::DEBUG::2012-12-09 12:20:33,946::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-737::WARNING::2012-12-09 12:20:33,948::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-737::DEBUG::2012-12-09 12:20:33,948::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-737::WARNING::2012-12-09 12:20:33,949::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-737::INFO::2012-12-09 12:20:33,950::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-737::DEBUG::2012-12-09 12:20:33,950::task::1172::TaskManager.Task::(prepare) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-737::DEBUG::2012-12-09 12:20:33,951::task::588::TaskManager.Task::(_updateState) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::moving from state preparing -> state finished
>Thread-737::DEBUG::2012-12-09 12:20:33,951::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-737::DEBUG::2012-12-09 12:20:33,951::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-737::DEBUG::2012-12-09 12:20:33,951::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-737::DEBUG::2012-12-09 12:20:33,952::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-737::DEBUG::2012-12-09 12:20:33,952::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-737::DEBUG::2012-12-09 12:20:33,952::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-737::DEBUG::2012-12-09 12:20:33,952::task::978::TaskManager.Task::(_decref) Task=`4338b7ef-4357-4467-b434-d4065ae2ea1f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:35,145::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:35,195::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025135 s, 40.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:36,561::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:36,577::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000800406 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:37,202::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:37,245::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01833 s, 55.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:39,251::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:39,295::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018441 s, 55.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:41,060::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:41,061::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:41,180::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:41,181::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:41,182::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:41,182::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:20:41,302::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:41,353::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270577 s, 37.8 MB/s\n'; <rc> = 0
>Thread-742::DEBUG::2012-12-09 12:20:42,345::task::588::TaskManager.Task::(_updateState) Task=`67d4b762-2c4b-45d7-9fac-78e17912f729`::moving from state init -> state preparing
>Thread-742::INFO::2012-12-09 12:20:42,345::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-742::INFO::2012-12-09 12:20:42,346::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169379711151', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-742::DEBUG::2012-12-09 12:20:42,346::task::1172::TaskManager.Task::(prepare) Task=`67d4b762-2c4b-45d7-9fac-78e17912f729`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169379711151', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-742::DEBUG::2012-12-09 12:20:42,347::task::588::TaskManager.Task::(_updateState) Task=`67d4b762-2c4b-45d7-9fac-78e17912f729`::moving from state preparing -> state finished
>Thread-742::DEBUG::2012-12-09 12:20:42,347::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-742::DEBUG::2012-12-09 12:20:42,348::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-742::DEBUG::2012-12-09 12:20:42,348::task::978::TaskManager.Task::(_decref) Task=`67d4b762-2c4b-45d7-9fac-78e17912f729`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:43,359::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:43,401::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190685 s, 53.7 MB/s\n'; <rc> = 0
>Thread-744::DEBUG::2012-12-09 12:20:44,114::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-744::DEBUG::2012-12-09 12:20:44,115::task::588::TaskManager.Task::(_updateState) Task=`dc4c273e-4891-4de7-8320-ee2c155ddb0f`::moving from state init -> state preparing
>Thread-744::INFO::2012-12-09 12:20:44,115::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-744::INFO::2012-12-09 12:20:44,116::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-744::DEBUG::2012-12-09 12:20:44,116::task::1172::TaskManager.Task::(prepare) Task=`dc4c273e-4891-4de7-8320-ee2c155ddb0f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-744::DEBUG::2012-12-09 12:20:44,117::task::588::TaskManager.Task::(_updateState) Task=`dc4c273e-4891-4de7-8320-ee2c155ddb0f`::moving from state preparing -> state finished
>Thread-744::DEBUG::2012-12-09 12:20:44,117::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-744::DEBUG::2012-12-09 12:20:44,117::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-744::DEBUG::2012-12-09 12:20:44,118::task::978::TaskManager.Task::(_decref) Task=`dc4c273e-4891-4de7-8320-ee2c155ddb0f`::ref 0 aborting False
>Thread-745::DEBUG::2012-12-09 12:20:44,142::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-745::DEBUG::2012-12-09 12:20:44,143::task::588::TaskManager.Task::(_updateState) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::moving from state init -> state preparing
>Thread-745::INFO::2012-12-09 12:20:44,143::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-745::DEBUG::2012-12-09 12:20:44,144::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d22b0b09-bf49-46b9-b4aa-923f5cdab9a7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-745::DEBUG::2012-12-09 12:20:44,144::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-745::DEBUG::2012-12-09 12:20:44,145::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-745::DEBUG::2012-12-09 12:20:44,145::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d22b0b09-bf49-46b9-b4aa-923f5cdab9a7`::Granted request
>Thread-745::DEBUG::2012-12-09 12:20:44,146::task::817::TaskManager.Task::(resourceAcquired) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-745::DEBUG::2012-12-09 12:20:44,146::task::978::TaskManager.Task::(_decref) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::ref 1 aborting False
>Thread-745::DEBUG::2012-12-09 12:20:44,148::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-745::DEBUG::2012-12-09 12:20:44,149::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-745::DEBUG::2012-12-09 12:20:44,261::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-745::WARNING::2012-12-09 12:20:44,262::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-745::DEBUG::2012-12-09 12:20:44,262::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-745::WARNING::2012-12-09 12:20:44,263::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-745::INFO::2012-12-09 12:20:44,263::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-745::DEBUG::2012-12-09 12:20:44,263::task::1172::TaskManager.Task::(prepare) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-745::DEBUG::2012-12-09 12:20:44,263::task::588::TaskManager.Task::(_updateState) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::moving from state preparing -> state finished
>Thread-745::DEBUG::2012-12-09 12:20:44,264::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-745::DEBUG::2012-12-09 12:20:44,264::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-745::DEBUG::2012-12-09 12:20:44,264::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-745::DEBUG::2012-12-09 12:20:44,264::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-745::DEBUG::2012-12-09 12:20:44,265::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-745::DEBUG::2012-12-09 12:20:44,265::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-745::DEBUG::2012-12-09 12:20:44,265::task::978::TaskManager.Task::(_decref) Task=`fdcf9aef-0883-4fc3-8ef7-374e87b4952c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:45,408::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:45,451::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198765 s, 51.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:46,582::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:46,593::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000669856 s, 6.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:47,458::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:47,506::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233529 s, 43.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:49,513::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:49,559::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209914 s, 48.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:20:51,183::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:20:51,184::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:20:51,303::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:20:51,305::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:20:51,306::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:20:51,307::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:20:51,566::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:51,614::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0231985 s, 44.1 MB/s\n'; <rc> = 0
>Thread-750::DEBUG::2012-12-09 12:20:52,630::task::588::TaskManager.Task::(_updateState) Task=`bc753275-c126-4b65-859e-088f003854a0`::moving from state init -> state preparing
>Thread-750::INFO::2012-12-09 12:20:52,631::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-750::INFO::2012-12-09 12:20:52,631::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011687040329', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-750::DEBUG::2012-12-09 12:20:52,631::task::1172::TaskManager.Task::(prepare) Task=`bc753275-c126-4b65-859e-088f003854a0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011687040329', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-750::DEBUG::2012-12-09 12:20:52,631::task::588::TaskManager.Task::(_updateState) Task=`bc753275-c126-4b65-859e-088f003854a0`::moving from state preparing -> state finished
>Thread-750::DEBUG::2012-12-09 12:20:52,632::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-750::DEBUG::2012-12-09 12:20:52,632::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-750::DEBUG::2012-12-09 12:20:52,632::task::978::TaskManager.Task::(_decref) Task=`bc753275-c126-4b65-859e-088f003854a0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:53,621::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:53,673::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273332 s, 37.5 MB/s\n'; <rc> = 0
>Thread-752::DEBUG::2012-12-09 12:20:54,402::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-752::DEBUG::2012-12-09 12:20:54,403::task::588::TaskManager.Task::(_updateState) Task=`8fa7af2c-2e33-4be7-9d8c-dd0a0be79313`::moving from state init -> state preparing
>Thread-752::INFO::2012-12-09 12:20:54,404::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-752::INFO::2012-12-09 12:20:54,404::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-752::DEBUG::2012-12-09 12:20:54,405::task::1172::TaskManager.Task::(prepare) Task=`8fa7af2c-2e33-4be7-9d8c-dd0a0be79313`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-752::DEBUG::2012-12-09 12:20:54,405::task::588::TaskManager.Task::(_updateState) Task=`8fa7af2c-2e33-4be7-9d8c-dd0a0be79313`::moving from state preparing -> state finished
>Thread-752::DEBUG::2012-12-09 12:20:54,406::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-752::DEBUG::2012-12-09 12:20:54,406::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-752::DEBUG::2012-12-09 12:20:54,407::task::978::TaskManager.Task::(_decref) Task=`8fa7af2c-2e33-4be7-9d8c-dd0a0be79313`::ref 0 aborting False
>Thread-753::DEBUG::2012-12-09 12:20:54,431::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-753::DEBUG::2012-12-09 12:20:54,432::task::588::TaskManager.Task::(_updateState) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::moving from state init -> state preparing
>Thread-753::INFO::2012-12-09 12:20:54,432::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-753::DEBUG::2012-12-09 12:20:54,433::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f051a70-8f12-46ba-91f9-fa55607a13d6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-753::DEBUG::2012-12-09 12:20:54,433::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-753::DEBUG::2012-12-09 12:20:54,434::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-753::DEBUG::2012-12-09 12:20:54,434::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f051a70-8f12-46ba-91f9-fa55607a13d6`::Granted request
>Thread-753::DEBUG::2012-12-09 12:20:54,435::task::817::TaskManager.Task::(resourceAcquired) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-753::DEBUG::2012-12-09 12:20:54,435::task::978::TaskManager.Task::(_decref) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::ref 1 aborting False
>Thread-753::DEBUG::2012-12-09 12:20:54,437::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-753::DEBUG::2012-12-09 12:20:54,438::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-753::DEBUG::2012-12-09 12:20:54,552::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-753::WARNING::2012-12-09 12:20:54,555::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-753::DEBUG::2012-12-09 12:20:54,555::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-753::WARNING::2012-12-09 12:20:54,556::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-753::INFO::2012-12-09 12:20:54,557::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-753::DEBUG::2012-12-09 12:20:54,557::task::1172::TaskManager.Task::(prepare) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-753::DEBUG::2012-12-09 12:20:54,558::task::588::TaskManager.Task::(_updateState) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::moving from state preparing -> state finished
>Thread-753::DEBUG::2012-12-09 12:20:54,558::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-753::DEBUG::2012-12-09 12:20:54,559::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-753::DEBUG::2012-12-09 12:20:54,559::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-753::DEBUG::2012-12-09 12:20:54,560::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-753::DEBUG::2012-12-09 12:20:54,560::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-753::DEBUG::2012-12-09 12:20:54,560::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-753::DEBUG::2012-12-09 12:20:54,561::task::978::TaskManager.Task::(_decref) Task=`9ebd5c96-e1be-4084-b0a9-a4dc9d4f5f2b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:20:55,680::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:55,727::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223248 s, 45.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:20:56,596::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:20:56,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000957795 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:57,734::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:57,786::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274013 s, 37.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:20:59,793::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:20:59,857::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0381647 s, 26.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:01,308::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:01,309::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:01,422::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:01,423::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:01,423::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:01,423::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:01,864::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:01,916::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263336 s, 38.9 MB/s\n'; <rc> = 0
>Thread-758::DEBUG::2012-12-09 12:21:02,839::task::588::TaskManager.Task::(_updateState) Task=`96d64eae-e228-439c-a918-4a12006725d0`::moving from state init -> state preparing
>Thread-758::INFO::2012-12-09 12:21:02,840::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-758::INFO::2012-12-09 12:21:02,840::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149989128113', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-758::DEBUG::2012-12-09 12:21:02,841::task::1172::TaskManager.Task::(prepare) Task=`96d64eae-e228-439c-a918-4a12006725d0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149989128113', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-758::DEBUG::2012-12-09 12:21:02,841::task::588::TaskManager.Task::(_updateState) Task=`96d64eae-e228-439c-a918-4a12006725d0`::moving from state preparing -> state finished
>Thread-758::DEBUG::2012-12-09 12:21:02,841::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-758::DEBUG::2012-12-09 12:21:02,842::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-758::DEBUG::2012-12-09 12:21:02,842::task::978::TaskManager.Task::(_decref) Task=`96d64eae-e228-439c-a918-4a12006725d0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:03,922::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:03,974::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267952 s, 38.2 MB/s\n'; <rc> = 0
>Thread-760::DEBUG::2012-12-09 12:21:04,632::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-760::DEBUG::2012-12-09 12:21:04,633::task::588::TaskManager.Task::(_updateState) Task=`9adc966b-1bc8-4a18-a512-2bc39c25d80b`::moving from state init -> state preparing
>Thread-760::INFO::2012-12-09 12:21:04,633::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-760::INFO::2012-12-09 12:21:04,633::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-760::DEBUG::2012-12-09 12:21:04,633::task::1172::TaskManager.Task::(prepare) Task=`9adc966b-1bc8-4a18-a512-2bc39c25d80b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-760::DEBUG::2012-12-09 12:21:04,634::task::588::TaskManager.Task::(_updateState) Task=`9adc966b-1bc8-4a18-a512-2bc39c25d80b`::moving from state preparing -> state finished
>Thread-760::DEBUG::2012-12-09 12:21:04,634::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-760::DEBUG::2012-12-09 12:21:04,634::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-760::DEBUG::2012-12-09 12:21:04,634::task::978::TaskManager.Task::(_decref) Task=`9adc966b-1bc8-4a18-a512-2bc39c25d80b`::ref 0 aborting False
>Thread-761::DEBUG::2012-12-09 12:21:04,650::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-761::DEBUG::2012-12-09 12:21:04,650::task::588::TaskManager.Task::(_updateState) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::moving from state init -> state preparing
>Thread-761::INFO::2012-12-09 12:21:04,651::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-761::DEBUG::2012-12-09 12:21:04,651::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ccc87822-097f-4f7c-b5f2-3500fa5267e5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-761::DEBUG::2012-12-09 12:21:04,651::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-761::DEBUG::2012-12-09 12:21:04,651::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-761::DEBUG::2012-12-09 12:21:04,652::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ccc87822-097f-4f7c-b5f2-3500fa5267e5`::Granted request
>Thread-761::DEBUG::2012-12-09 12:21:04,652::task::817::TaskManager.Task::(resourceAcquired) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-761::DEBUG::2012-12-09 12:21:04,652::task::978::TaskManager.Task::(_decref) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::ref 1 aborting False
>Thread-761::DEBUG::2012-12-09 12:21:04,653::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-761::DEBUG::2012-12-09 12:21:04,653::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-761::DEBUG::2012-12-09 12:21:04,761::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-761::WARNING::2012-12-09 12:21:04,762::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-761::DEBUG::2012-12-09 12:21:04,762::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-761::WARNING::2012-12-09 12:21:04,763::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-761::INFO::2012-12-09 12:21:04,763::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-761::DEBUG::2012-12-09 12:21:04,763::task::1172::TaskManager.Task::(prepare) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-761::DEBUG::2012-12-09 12:21:04,764::task::588::TaskManager.Task::(_updateState) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::moving from state preparing -> state finished
>Thread-761::DEBUG::2012-12-09 12:21:04,764::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-761::DEBUG::2012-12-09 12:21:04,764::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-761::DEBUG::2012-12-09 12:21:04,764::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-761::DEBUG::2012-12-09 12:21:04,765::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-761::DEBUG::2012-12-09 12:21:04,765::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-761::DEBUG::2012-12-09 12:21:04,765::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-761::DEBUG::2012-12-09 12:21:04,765::task::978::TaskManager.Task::(_decref) Task=`cab0ef85-4e4f-46fe-b9e5-c704941a86ae`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:05,981::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:06,027::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214666 s, 47.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:21:06,614::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:06,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000881994 s, 4.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:08,033::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:08,071::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153495 s, 66.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:10,078::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:10,127::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235527 s, 43.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:11,425::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:11,426::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:11,542::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:11,543::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:11,543::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:11,543::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:12,134::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:12,184::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259002 s, 39.5 MB/s\n'; <rc> = 0
>Thread-766::DEBUG::2012-12-09 12:21:13,078::task::588::TaskManager.Task::(_updateState) Task=`77fc4ace-4693-4764-85ae-a60ed438f87d`::moving from state init -> state preparing
>Thread-766::INFO::2012-12-09 12:21:13,079::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-766::INFO::2012-12-09 12:21:13,080::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153849124908', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-766::DEBUG::2012-12-09 12:21:13,080::task::1172::TaskManager.Task::(prepare) Task=`77fc4ace-4693-4764-85ae-a60ed438f87d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153849124908', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-766::DEBUG::2012-12-09 12:21:13,080::task::588::TaskManager.Task::(_updateState) Task=`77fc4ace-4693-4764-85ae-a60ed438f87d`::moving from state preparing -> state finished
>Thread-766::DEBUG::2012-12-09 12:21:13,081::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-766::DEBUG::2012-12-09 12:21:13,081::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-766::DEBUG::2012-12-09 12:21:13,081::task::978::TaskManager.Task::(_decref) Task=`77fc4ace-4693-4764-85ae-a60ed438f87d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:14,191::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:14,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273204 s, 37.5 MB/s\n'; <rc> = 0
>Thread-768::DEBUG::2012-12-09 12:21:14,892::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-768::DEBUG::2012-12-09 12:21:14,893::task::588::TaskManager.Task::(_updateState) Task=`ca27b920-b514-4eff-b92b-fcda5cdafbe2`::moving from state init -> state preparing
>Thread-768::INFO::2012-12-09 12:21:14,893::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-768::INFO::2012-12-09 12:21:14,894::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-768::DEBUG::2012-12-09 12:21:14,895::task::1172::TaskManager.Task::(prepare) Task=`ca27b920-b514-4eff-b92b-fcda5cdafbe2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-768::DEBUG::2012-12-09 12:21:14,896::task::588::TaskManager.Task::(_updateState) Task=`ca27b920-b514-4eff-b92b-fcda5cdafbe2`::moving from state preparing -> state finished
>Thread-768::DEBUG::2012-12-09 12:21:14,896::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-768::DEBUG::2012-12-09 12:21:14,897::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-768::DEBUG::2012-12-09 12:21:14,898::task::978::TaskManager.Task::(_decref) Task=`ca27b920-b514-4eff-b92b-fcda5cdafbe2`::ref 0 aborting False
>Thread-769::DEBUG::2012-12-09 12:21:14,915::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-769::DEBUG::2012-12-09 12:21:14,915::task::588::TaskManager.Task::(_updateState) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::moving from state init -> state preparing
>Thread-769::INFO::2012-12-09 12:21:14,915::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-769::DEBUG::2012-12-09 12:21:14,916::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e7ba5343-3a7e-418f-9a68-ede26e71bc6b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-769::DEBUG::2012-12-09 12:21:14,916::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-769::DEBUG::2012-12-09 12:21:14,916::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-769::DEBUG::2012-12-09 12:21:14,916::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e7ba5343-3a7e-418f-9a68-ede26e71bc6b`::Granted request
>Thread-769::DEBUG::2012-12-09 12:21:14,917::task::817::TaskManager.Task::(resourceAcquired) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-769::DEBUG::2012-12-09 12:21:14,917::task::978::TaskManager.Task::(_decref) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::ref 1 aborting False
>Thread-769::DEBUG::2012-12-09 12:21:14,918::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-769::DEBUG::2012-12-09 12:21:14,918::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-769::DEBUG::2012-12-09 12:21:15,014::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-769::WARNING::2012-12-09 12:21:15,015::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-769::DEBUG::2012-12-09 12:21:15,016::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-769::WARNING::2012-12-09 12:21:15,016::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-769::INFO::2012-12-09 12:21:15,017::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-769::DEBUG::2012-12-09 12:21:15,017::task::1172::TaskManager.Task::(prepare) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-769::DEBUG::2012-12-09 12:21:15,017::task::588::TaskManager.Task::(_updateState) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::moving from state preparing -> state finished
>Thread-769::DEBUG::2012-12-09 12:21:15,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-769::DEBUG::2012-12-09 12:21:15,018::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-769::DEBUG::2012-12-09 12:21:15,018::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-769::DEBUG::2012-12-09 12:21:15,018::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-769::DEBUG::2012-12-09 12:21:15,018::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-769::DEBUG::2012-12-09 12:21:15,018::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-769::DEBUG::2012-12-09 12:21:15,019::task::978::TaskManager.Task::(_decref) Task=`aedb29fa-e2bc-4438-ba59-aaae00b823ec`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:16,251::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:16,303::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278632 s, 36.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:21:16,633::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:16,648::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000726388 s, 5.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:18,310::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:18,359::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228416 s, 44.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:20,366::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:20,408::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169853 s, 60.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:21,544::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:21,546::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:21,654::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:21,656::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:21,656::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:21,657::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:22,415::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:22,466::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249941 s, 41.0 MB/s\n'; <rc> = 0
>Thread-774::DEBUG::2012-12-09 12:21:23,429::task::588::TaskManager.Task::(_updateState) Task=`3388df8a-2a18-4555-9ef8-ebef11909ca0`::moving from state init -> state preparing
>Thread-774::INFO::2012-12-09 12:21:23,429::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-774::INFO::2012-12-09 12:21:23,429::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164220333099', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-774::DEBUG::2012-12-09 12:21:23,430::task::1172::TaskManager.Task::(prepare) Task=`3388df8a-2a18-4555-9ef8-ebef11909ca0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164220333099', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-774::DEBUG::2012-12-09 12:21:23,430::task::588::TaskManager.Task::(_updateState) Task=`3388df8a-2a18-4555-9ef8-ebef11909ca0`::moving from state preparing -> state finished
>Thread-774::DEBUG::2012-12-09 12:21:23,430::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-774::DEBUG::2012-12-09 12:21:23,430::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-774::DEBUG::2012-12-09 12:21:23,430::task::978::TaskManager.Task::(_decref) Task=`3388df8a-2a18-4555-9ef8-ebef11909ca0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:24,473::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:24,524::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266673 s, 38.4 MB/s\n'; <rc> = 0
>Thread-776::DEBUG::2012-12-09 12:21:25,124::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-776::DEBUG::2012-12-09 12:21:25,125::task::588::TaskManager.Task::(_updateState) Task=`0e69a134-5d85-47b4-bbb7-864202bdade7`::moving from state init -> state preparing
>Thread-776::INFO::2012-12-09 12:21:25,125::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-776::INFO::2012-12-09 12:21:25,126::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-776::DEBUG::2012-12-09 12:21:25,126::task::1172::TaskManager.Task::(prepare) Task=`0e69a134-5d85-47b4-bbb7-864202bdade7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-776::DEBUG::2012-12-09 12:21:25,127::task::588::TaskManager.Task::(_updateState) Task=`0e69a134-5d85-47b4-bbb7-864202bdade7`::moving from state preparing -> state finished
>Thread-776::DEBUG::2012-12-09 12:21:25,127::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-776::DEBUG::2012-12-09 12:21:25,128::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-776::DEBUG::2012-12-09 12:21:25,128::task::978::TaskManager.Task::(_decref) Task=`0e69a134-5d85-47b4-bbb7-864202bdade7`::ref 0 aborting False
>Thread-777::DEBUG::2012-12-09 12:21:25,147::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-777::DEBUG::2012-12-09 12:21:25,148::task::588::TaskManager.Task::(_updateState) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::moving from state init -> state preparing
>Thread-777::INFO::2012-12-09 12:21:25,148::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-777::DEBUG::2012-12-09 12:21:25,149::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a90aec2-c5d4-4992-87eb-0f051209748a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-777::DEBUG::2012-12-09 12:21:25,149::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-777::DEBUG::2012-12-09 12:21:25,150::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-777::DEBUG::2012-12-09 12:21:25,150::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a90aec2-c5d4-4992-87eb-0f051209748a`::Granted request
>Thread-777::DEBUG::2012-12-09 12:21:25,151::task::817::TaskManager.Task::(resourceAcquired) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-777::DEBUG::2012-12-09 12:21:25,151::task::978::TaskManager.Task::(_decref) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::ref 1 aborting False
>Thread-777::DEBUG::2012-12-09 12:21:25,153::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-777::DEBUG::2012-12-09 12:21:25,154::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-777::DEBUG::2012-12-09 12:21:25,261::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-777::WARNING::2012-12-09 12:21:25,262::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-777::DEBUG::2012-12-09 12:21:25,262::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-777::WARNING::2012-12-09 12:21:25,263::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-777::INFO::2012-12-09 12:21:25,263::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-777::DEBUG::2012-12-09 12:21:25,263::task::1172::TaskManager.Task::(prepare) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-777::DEBUG::2012-12-09 12:21:25,264::task::588::TaskManager.Task::(_updateState) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::moving from state preparing -> state finished
>Thread-777::DEBUG::2012-12-09 12:21:25,264::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-777::DEBUG::2012-12-09 12:21:25,264::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-777::DEBUG::2012-12-09 12:21:25,264::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-777::DEBUG::2012-12-09 12:21:25,265::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-777::DEBUG::2012-12-09 12:21:25,265::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-777::DEBUG::2012-12-09 12:21:25,265::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-777::DEBUG::2012-12-09 12:21:25,265::task::978::TaskManager.Task::(_decref) Task=`26518cdd-bbb6-449d-b16d-cf89ebb05341`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:26,531::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:26,584::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027764 s, 36.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:21:26,653::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:26,668::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00088265 s, 4.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:28,591::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:28,643::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280128 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:30,650::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:30,697::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224685 s, 45.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:31,657::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:31,660::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:31,771::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:31,772::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:31,772::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:31,772::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:32,703::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:32,747::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196587 s, 52.1 MB/s\n'; <rc> = 0
>Thread-782::DEBUG::2012-12-09 12:21:33,689::task::588::TaskManager.Task::(_updateState) Task=`7e33a0eb-ddd4-4a0e-8238-a777cd5f1807`::moving from state init -> state preparing
>Thread-782::INFO::2012-12-09 12:21:33,689::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-782::INFO::2012-12-09 12:21:33,689::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167419910431', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-782::DEBUG::2012-12-09 12:21:33,690::task::1172::TaskManager.Task::(prepare) Task=`7e33a0eb-ddd4-4a0e-8238-a777cd5f1807`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167419910431', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-782::DEBUG::2012-12-09 12:21:33,690::task::588::TaskManager.Task::(_updateState) Task=`7e33a0eb-ddd4-4a0e-8238-a777cd5f1807`::moving from state preparing -> state finished
>Thread-782::DEBUG::2012-12-09 12:21:33,690::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-782::DEBUG::2012-12-09 12:21:33,691::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-782::DEBUG::2012-12-09 12:21:33,691::task::978::TaskManager.Task::(_decref) Task=`7e33a0eb-ddd4-4a0e-8238-a777cd5f1807`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:34,753::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:34,804::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260205 s, 39.4 MB/s\n'; <rc> = 0
>Thread-784::DEBUG::2012-12-09 12:21:35,412::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-784::DEBUG::2012-12-09 12:21:35,413::task::588::TaskManager.Task::(_updateState) Task=`cf0b4a27-f22e-424d-ba6e-41c1da7b06db`::moving from state init -> state preparing
>Thread-784::INFO::2012-12-09 12:21:35,414::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-784::INFO::2012-12-09 12:21:35,414::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-784::DEBUG::2012-12-09 12:21:35,415::task::1172::TaskManager.Task::(prepare) Task=`cf0b4a27-f22e-424d-ba6e-41c1da7b06db`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-784::DEBUG::2012-12-09 12:21:35,415::task::588::TaskManager.Task::(_updateState) Task=`cf0b4a27-f22e-424d-ba6e-41c1da7b06db`::moving from state preparing -> state finished
>Thread-784::DEBUG::2012-12-09 12:21:35,416::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-784::DEBUG::2012-12-09 12:21:35,416::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-784::DEBUG::2012-12-09 12:21:35,417::task::978::TaskManager.Task::(_decref) Task=`cf0b4a27-f22e-424d-ba6e-41c1da7b06db`::ref 0 aborting False
>Thread-785::DEBUG::2012-12-09 12:21:35,442::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-785::DEBUG::2012-12-09 12:21:35,443::task::588::TaskManager.Task::(_updateState) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::moving from state init -> state preparing
>Thread-785::INFO::2012-12-09 12:21:35,443::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-785::DEBUG::2012-12-09 12:21:35,444::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`268b8bec-7f87-44e8-af4e-5a9f81fcf5cd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-785::DEBUG::2012-12-09 12:21:35,444::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-785::DEBUG::2012-12-09 12:21:35,445::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-785::DEBUG::2012-12-09 12:21:35,445::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`268b8bec-7f87-44e8-af4e-5a9f81fcf5cd`::Granted request
>Thread-785::DEBUG::2012-12-09 12:21:35,446::task::817::TaskManager.Task::(resourceAcquired) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-785::DEBUG::2012-12-09 12:21:35,446::task::978::TaskManager.Task::(_decref) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::ref 1 aborting False
>Thread-785::DEBUG::2012-12-09 12:21:35,448::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-785::DEBUG::2012-12-09 12:21:35,449::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-785::DEBUG::2012-12-09 12:21:35,553::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-785::WARNING::2012-12-09 12:21:35,555::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-785::DEBUG::2012-12-09 12:21:35,556::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-785::WARNING::2012-12-09 12:21:35,556::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-785::INFO::2012-12-09 12:21:35,557::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-785::DEBUG::2012-12-09 12:21:35,558::task::1172::TaskManager.Task::(prepare) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-785::DEBUG::2012-12-09 12:21:35,558::task::588::TaskManager.Task::(_updateState) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::moving from state preparing -> state finished
>Thread-785::DEBUG::2012-12-09 12:21:35,559::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-785::DEBUG::2012-12-09 12:21:35,559::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-785::DEBUG::2012-12-09 12:21:35,560::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-785::DEBUG::2012-12-09 12:21:35,560::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-785::DEBUG::2012-12-09 12:21:35,560::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-785::DEBUG::2012-12-09 12:21:35,561::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-785::DEBUG::2012-12-09 12:21:35,561::task::978::TaskManager.Task::(_decref) Task=`6db4c77e-a1c5-4bc6-b06c-63d9b7643064`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:21:36,673::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:36,685::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000945607 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:36,811::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:36,857::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216699 s, 47.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:38,864::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:38,916::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258934 s, 39.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:40,923::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:40,976::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270913 s, 37.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:41,773::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:41,775::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:41,893::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:41,894::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:41,894::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:41,894::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:42,982::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:43,027::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211242 s, 48.5 MB/s\n'; <rc> = 0
>Thread-790::DEBUG::2012-12-09 12:21:43,980::task::588::TaskManager.Task::(_updateState) Task=`30ee5109-4536-4aaf-b51c-986dae527df4`::moving from state init -> state preparing
>Thread-790::INFO::2012-12-09 12:21:43,981::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-790::INFO::2012-12-09 12:21:43,982::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0139851570129', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.1', 'code': 358, 'valid': False}}
>Thread-790::DEBUG::2012-12-09 12:21:43,982::task::1172::TaskManager.Task::(prepare) Task=`30ee5109-4536-4aaf-b51c-986dae527df4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0139851570129', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.1', 'code': 358, 'valid': False}}
>Thread-790::DEBUG::2012-12-09 12:21:43,982::task::588::TaskManager.Task::(_updateState) Task=`30ee5109-4536-4aaf-b51c-986dae527df4`::moving from state preparing -> state finished
>Thread-790::DEBUG::2012-12-09 12:21:43,983::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-790::DEBUG::2012-12-09 12:21:43,983::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-790::DEBUG::2012-12-09 12:21:43,983::task::978::TaskManager.Task::(_decref) Task=`30ee5109-4536-4aaf-b51c-986dae527df4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:45,034::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:45,086::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026246 s, 39.0 MB/s\n'; <rc> = 0
>Thread-792::DEBUG::2012-12-09 12:21:45,736::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-792::DEBUG::2012-12-09 12:21:45,736::task::588::TaskManager.Task::(_updateState) Task=`ca0e54a0-c475-4091-9064-9c84b2c742af`::moving from state init -> state preparing
>Thread-792::INFO::2012-12-09 12:21:45,737::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-792::INFO::2012-12-09 12:21:45,738::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-792::DEBUG::2012-12-09 12:21:45,738::task::1172::TaskManager.Task::(prepare) Task=`ca0e54a0-c475-4091-9064-9c84b2c742af`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-792::DEBUG::2012-12-09 12:21:45,738::task::588::TaskManager.Task::(_updateState) Task=`ca0e54a0-c475-4091-9064-9c84b2c742af`::moving from state preparing -> state finished
>Thread-792::DEBUG::2012-12-09 12:21:45,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-792::DEBUG::2012-12-09 12:21:45,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-792::DEBUG::2012-12-09 12:21:45,740::task::978::TaskManager.Task::(_decref) Task=`ca0e54a0-c475-4091-9064-9c84b2c742af`::ref 0 aborting False
>Thread-793::DEBUG::2012-12-09 12:21:45,756::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-793::DEBUG::2012-12-09 12:21:45,757::task::588::TaskManager.Task::(_updateState) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::moving from state init -> state preparing
>Thread-793::INFO::2012-12-09 12:21:45,757::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-793::DEBUG::2012-12-09 12:21:45,758::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f1300ea8-aabd-4b83-b863-97cf6b9d3fbc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-793::DEBUG::2012-12-09 12:21:45,758::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-793::DEBUG::2012-12-09 12:21:45,759::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-793::DEBUG::2012-12-09 12:21:45,759::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f1300ea8-aabd-4b83-b863-97cf6b9d3fbc`::Granted request
>Thread-793::DEBUG::2012-12-09 12:21:45,760::task::817::TaskManager.Task::(resourceAcquired) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-793::DEBUG::2012-12-09 12:21:45,760::task::978::TaskManager.Task::(_decref) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::ref 1 aborting False
>Thread-793::DEBUG::2012-12-09 12:21:45,762::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-793::DEBUG::2012-12-09 12:21:45,763::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-793::DEBUG::2012-12-09 12:21:45,874::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-793::WARNING::2012-12-09 12:21:45,876::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-793::DEBUG::2012-12-09 12:21:45,876::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-793::WARNING::2012-12-09 12:21:45,877::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-793::INFO::2012-12-09 12:21:45,878::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-793::DEBUG::2012-12-09 12:21:45,879::task::1172::TaskManager.Task::(prepare) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-793::DEBUG::2012-12-09 12:21:45,879::task::588::TaskManager.Task::(_updateState) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::moving from state preparing -> state finished
>Thread-793::DEBUG::2012-12-09 12:21:45,880::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-793::DEBUG::2012-12-09 12:21:45,880::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-793::DEBUG::2012-12-09 12:21:45,880::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-793::DEBUG::2012-12-09 12:21:45,881::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-793::DEBUG::2012-12-09 12:21:45,881::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-793::DEBUG::2012-12-09 12:21:45,881::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-793::DEBUG::2012-12-09 12:21:45,881::task::978::TaskManager.Task::(_decref) Task=`9d959e7f-252a-47dc-bbd7-3a4ed9434e3e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:21:46,691::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:46,707::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000834179 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:47,092::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:47,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272914 s, 37.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:49,150::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:49,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221537 s, 46.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:51,204::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:51,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257907 s, 39.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:21:51,895::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:21:51,897::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:21:52,014::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:21:52,016::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:21:52,016::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:21:52,017::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:21:53,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:53,307::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217905 s, 47.0 MB/s\n'; <rc> = 0
>Thread-798::DEBUG::2012-12-09 12:21:54,285::task::588::TaskManager.Task::(_updateState) Task=`b13c158f-5f12-48c5-8deb-df8771b4131b`::moving from state init -> state preparing
>Thread-798::INFO::2012-12-09 12:21:54,285::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-798::INFO::2012-12-09 12:21:54,286::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168519020081', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-798::DEBUG::2012-12-09 12:21:54,286::task::1172::TaskManager.Task::(prepare) Task=`b13c158f-5f12-48c5-8deb-df8771b4131b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168519020081', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-798::DEBUG::2012-12-09 12:21:54,287::task::588::TaskManager.Task::(_updateState) Task=`b13c158f-5f12-48c5-8deb-df8771b4131b`::moving from state preparing -> state finished
>Thread-798::DEBUG::2012-12-09 12:21:54,287::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-798::DEBUG::2012-12-09 12:21:54,288::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-798::DEBUG::2012-12-09 12:21:54,288::task::978::TaskManager.Task::(_decref) Task=`b13c158f-5f12-48c5-8deb-df8771b4131b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:21:55,314::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:55,359::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211484 s, 48.4 MB/s\n'; <rc> = 0
>Thread-800::DEBUG::2012-12-09 12:21:55,954::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-800::DEBUG::2012-12-09 12:21:55,955::task::588::TaskManager.Task::(_updateState) Task=`8e31ff7a-08b1-4b88-973c-cae891864541`::moving from state init -> state preparing
>Thread-800::INFO::2012-12-09 12:21:55,956::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-800::INFO::2012-12-09 12:21:55,956::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-800::DEBUG::2012-12-09 12:21:55,957::task::1172::TaskManager.Task::(prepare) Task=`8e31ff7a-08b1-4b88-973c-cae891864541`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-800::DEBUG::2012-12-09 12:21:55,957::task::588::TaskManager.Task::(_updateState) Task=`8e31ff7a-08b1-4b88-973c-cae891864541`::moving from state preparing -> state finished
>Thread-800::DEBUG::2012-12-09 12:21:55,957::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-800::DEBUG::2012-12-09 12:21:55,958::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-800::DEBUG::2012-12-09 12:21:55,958::task::978::TaskManager.Task::(_decref) Task=`8e31ff7a-08b1-4b88-973c-cae891864541`::ref 0 aborting False
>Thread-801::DEBUG::2012-12-09 12:21:55,977::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-801::DEBUG::2012-12-09 12:21:55,978::task::588::TaskManager.Task::(_updateState) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::moving from state init -> state preparing
>Thread-801::INFO::2012-12-09 12:21:55,978::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-801::DEBUG::2012-12-09 12:21:55,978::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`126ddf4c-d66c-4882-b7bd-d37af5fa8c06`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-801::DEBUG::2012-12-09 12:21:55,979::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-801::DEBUG::2012-12-09 12:21:55,979::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-801::DEBUG::2012-12-09 12:21:55,979::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`126ddf4c-d66c-4882-b7bd-d37af5fa8c06`::Granted request
>Thread-801::DEBUG::2012-12-09 12:21:55,979::task::817::TaskManager.Task::(resourceAcquired) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-801::DEBUG::2012-12-09 12:21:55,980::task::978::TaskManager.Task::(_decref) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::ref 1 aborting False
>Thread-801::DEBUG::2012-12-09 12:21:55,980::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-801::DEBUG::2012-12-09 12:21:55,981::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-801::DEBUG::2012-12-09 12:21:56,089::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-801::WARNING::2012-12-09 12:21:56,091::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-801::DEBUG::2012-12-09 12:21:56,091::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-801::WARNING::2012-12-09 12:21:56,091::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-801::INFO::2012-12-09 12:21:56,092::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-801::DEBUG::2012-12-09 12:21:56,092::task::1172::TaskManager.Task::(prepare) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-801::DEBUG::2012-12-09 12:21:56,092::task::588::TaskManager.Task::(_updateState) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::moving from state preparing -> state finished
>Thread-801::DEBUG::2012-12-09 12:21:56,092::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-801::DEBUG::2012-12-09 12:21:56,092::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-801::DEBUG::2012-12-09 12:21:56,093::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-801::DEBUG::2012-12-09 12:21:56,093::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-801::DEBUG::2012-12-09 12:21:56,093::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-801::DEBUG::2012-12-09 12:21:56,093::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-801::DEBUG::2012-12-09 12:21:56,094::task::978::TaskManager.Task::(_decref) Task=`3406d802-9c4d-4f51-89ad-68792ba7b5d6`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:21:56,712::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:21:56,736::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000723844 s, 5.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:57,366::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:57,415::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247208 s, 41.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:21:59,421::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:21:59,469::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227452 s, 45.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:01,476::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:01,526::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250493 s, 40.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:02,019::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:02,020::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:02,125::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:02,126::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:02,126::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:02,127::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:03,532::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:03,583::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212361 s, 48.2 MB/s\n'; <rc> = 0
>Thread-806::DEBUG::2012-12-09 12:22:04,637::task::588::TaskManager.Task::(_updateState) Task=`29eebb54-fbcc-4edc-98fc-8755a3540590`::moving from state init -> state preparing
>Thread-806::INFO::2012-12-09 12:22:04,638::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-806::INFO::2012-12-09 12:22:04,638::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0250380039215', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-806::DEBUG::2012-12-09 12:22:04,638::task::1172::TaskManager.Task::(prepare) Task=`29eebb54-fbcc-4edc-98fc-8755a3540590`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0250380039215', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-806::DEBUG::2012-12-09 12:22:04,638::task::588::TaskManager.Task::(_updateState) Task=`29eebb54-fbcc-4edc-98fc-8755a3540590`::moving from state preparing -> state finished
>Thread-806::DEBUG::2012-12-09 12:22:04,639::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-806::DEBUG::2012-12-09 12:22:04,639::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-806::DEBUG::2012-12-09 12:22:04,639::task::978::TaskManager.Task::(_decref) Task=`29eebb54-fbcc-4edc-98fc-8755a3540590`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:05,590::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:05,638::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237313 s, 43.1 MB/s\n'; <rc> = 0
>Thread-808::DEBUG::2012-12-09 12:22:06,258::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-808::DEBUG::2012-12-09 12:22:06,258::task::588::TaskManager.Task::(_updateState) Task=`3991fd04-f15d-47b9-9072-dae3649c0e6b`::moving from state init -> state preparing
>Thread-808::INFO::2012-12-09 12:22:06,258::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-808::INFO::2012-12-09 12:22:06,259::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-808::DEBUG::2012-12-09 12:22:06,259::task::1172::TaskManager.Task::(prepare) Task=`3991fd04-f15d-47b9-9072-dae3649c0e6b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-808::DEBUG::2012-12-09 12:22:06,259::task::588::TaskManager.Task::(_updateState) Task=`3991fd04-f15d-47b9-9072-dae3649c0e6b`::moving from state preparing -> state finished
>Thread-808::DEBUG::2012-12-09 12:22:06,259::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-808::DEBUG::2012-12-09 12:22:06,260::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-808::DEBUG::2012-12-09 12:22:06,260::task::978::TaskManager.Task::(_decref) Task=`3991fd04-f15d-47b9-9072-dae3649c0e6b`::ref 0 aborting False
>Thread-809::DEBUG::2012-12-09 12:22:06,273::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-809::DEBUG::2012-12-09 12:22:06,274::task::588::TaskManager.Task::(_updateState) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::moving from state init -> state preparing
>Thread-809::INFO::2012-12-09 12:22:06,274::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-809::DEBUG::2012-12-09 12:22:06,274::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e69aab4f-e4f4-406f-9858-3d8ad91415fc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-809::DEBUG::2012-12-09 12:22:06,274::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-809::DEBUG::2012-12-09 12:22:06,275::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-809::DEBUG::2012-12-09 12:22:06,275::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e69aab4f-e4f4-406f-9858-3d8ad91415fc`::Granted request
>Thread-809::DEBUG::2012-12-09 12:22:06,276::task::817::TaskManager.Task::(resourceAcquired) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-809::DEBUG::2012-12-09 12:22:06,276::task::978::TaskManager.Task::(_decref) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::ref 1 aborting False
>Thread-809::DEBUG::2012-12-09 12:22:06,277::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-809::DEBUG::2012-12-09 12:22:06,277::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-809::DEBUG::2012-12-09 12:22:06,380::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-809::WARNING::2012-12-09 12:22:06,381::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-809::DEBUG::2012-12-09 12:22:06,381::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-809::WARNING::2012-12-09 12:22:06,382::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-809::INFO::2012-12-09 12:22:06,382::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-809::DEBUG::2012-12-09 12:22:06,382::task::1172::TaskManager.Task::(prepare) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-809::DEBUG::2012-12-09 12:22:06,383::task::588::TaskManager.Task::(_updateState) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::moving from state preparing -> state finished
>Thread-809::DEBUG::2012-12-09 12:22:06,383::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-809::DEBUG::2012-12-09 12:22:06,383::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-809::DEBUG::2012-12-09 12:22:06,383::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-809::DEBUG::2012-12-09 12:22:06,384::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-809::DEBUG::2012-12-09 12:22:06,384::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-809::DEBUG::2012-12-09 12:22:06,384::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-809::DEBUG::2012-12-09 12:22:06,384::task::978::TaskManager.Task::(_decref) Task=`7849566c-3785-49f0-bf68-a9955e89b6e7`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:22:06,740::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:06,752::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000672343 s, 6.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:07,645::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:07,695::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02595 s, 39.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:09,702::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:09,754::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274619 s, 37.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:11,760::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:11,808::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235896 s, 43.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:12,128::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:12,129::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:12,233::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:12,236::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:12,236::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:12,237::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:13,815::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:13,865::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248901 s, 41.1 MB/s\n'; <rc> = 0
>Thread-814::DEBUG::2012-12-09 12:22:14,928::task::588::TaskManager.Task::(_updateState) Task=`29bb3b1f-85fd-438a-a6a7-0ca50ee4e4c2`::moving from state init -> state preparing
>Thread-814::INFO::2012-12-09 12:22:14,929::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-814::INFO::2012-12-09 12:22:14,929::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0132369995117', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-814::DEBUG::2012-12-09 12:22:14,929::task::1172::TaskManager.Task::(prepare) Task=`29bb3b1f-85fd-438a-a6a7-0ca50ee4e4c2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0132369995117', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-814::DEBUG::2012-12-09 12:22:14,930::task::588::TaskManager.Task::(_updateState) Task=`29bb3b1f-85fd-438a-a6a7-0ca50ee4e4c2`::moving from state preparing -> state finished
>Thread-814::DEBUG::2012-12-09 12:22:14,930::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-814::DEBUG::2012-12-09 12:22:14,930::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-814::DEBUG::2012-12-09 12:22:14,931::task::978::TaskManager.Task::(_decref) Task=`29bb3b1f-85fd-438a-a6a7-0ca50ee4e4c2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:15,872::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:15,917::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210746 s, 48.6 MB/s\n'; <rc> = 0
>Thread-816::DEBUG::2012-12-09 12:22:16,492::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-816::DEBUG::2012-12-09 12:22:16,493::task::588::TaskManager.Task::(_updateState) Task=`915de2cd-9a19-4923-becb-4a0e140de600`::moving from state init -> state preparing
>Thread-816::INFO::2012-12-09 12:22:16,494::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-816::INFO::2012-12-09 12:22:16,494::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-816::DEBUG::2012-12-09 12:22:16,495::task::1172::TaskManager.Task::(prepare) Task=`915de2cd-9a19-4923-becb-4a0e140de600`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-816::DEBUG::2012-12-09 12:22:16,495::task::588::TaskManager.Task::(_updateState) Task=`915de2cd-9a19-4923-becb-4a0e140de600`::moving from state preparing -> state finished
>Thread-816::DEBUG::2012-12-09 12:22:16,495::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-816::DEBUG::2012-12-09 12:22:16,496::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-816::DEBUG::2012-12-09 12:22:16,496::task::978::TaskManager.Task::(_decref) Task=`915de2cd-9a19-4923-becb-4a0e140de600`::ref 0 aborting False
>Thread-817::DEBUG::2012-12-09 12:22:16,521::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-817::DEBUG::2012-12-09 12:22:16,522::task::588::TaskManager.Task::(_updateState) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::moving from state init -> state preparing
>Thread-817::INFO::2012-12-09 12:22:16,523::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-817::DEBUG::2012-12-09 12:22:16,523::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a6e79f0a-36f1-4f7b-b46e-57926d7d760c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-817::DEBUG::2012-12-09 12:22:16,524::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-817::DEBUG::2012-12-09 12:22:16,524::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-817::DEBUG::2012-12-09 12:22:16,525::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a6e79f0a-36f1-4f7b-b46e-57926d7d760c`::Granted request
>Thread-817::DEBUG::2012-12-09 12:22:16,525::task::817::TaskManager.Task::(resourceAcquired) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-817::DEBUG::2012-12-09 12:22:16,526::task::978::TaskManager.Task::(_decref) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::ref 1 aborting False
>Thread-817::DEBUG::2012-12-09 12:22:16,527::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-817::DEBUG::2012-12-09 12:22:16,528::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-817::DEBUG::2012-12-09 12:22:16,634::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-817::WARNING::2012-12-09 12:22:16,636::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-817::DEBUG::2012-12-09 12:22:16,636::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-817::WARNING::2012-12-09 12:22:16,637::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-817::INFO::2012-12-09 12:22:16,638::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-817::DEBUG::2012-12-09 12:22:16,638::task::1172::TaskManager.Task::(prepare) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-817::DEBUG::2012-12-09 12:22:16,639::task::588::TaskManager.Task::(_updateState) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::moving from state preparing -> state finished
>Thread-817::DEBUG::2012-12-09 12:22:16,639::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-817::DEBUG::2012-12-09 12:22:16,640::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-817::DEBUG::2012-12-09 12:22:16,640::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-817::DEBUG::2012-12-09 12:22:16,641::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-817::DEBUG::2012-12-09 12:22:16,641::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-817::DEBUG::2012-12-09 12:22:16,641::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-817::DEBUG::2012-12-09 12:22:16,641::task::978::TaskManager.Task::(_decref) Task=`864ac7d4-2d40-4ab1-98ad-92d8e3eef7e6`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:22:16,756::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:16,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000580957 s, 7.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:17,924::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:17,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267544 s, 38.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:19,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:20,031::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020606 s, 49.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:22,038::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:22,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243636 s, 42.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:22,239::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:22,240::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:22,348::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:22,350::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:22,351::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:22,351::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:24,093::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:24,144::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260668 s, 39.3 MB/s\n'; <rc> = 0
>Thread-822::DEBUG::2012-12-09 12:22:25,242::task::588::TaskManager.Task::(_updateState) Task=`768cd67e-8b36-4e98-849a-5ebcbb03ca0c`::moving from state init -> state preparing
>Thread-822::INFO::2012-12-09 12:22:25,243::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-822::INFO::2012-12-09 12:22:25,243::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112380981445', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-822::DEBUG::2012-12-09 12:22:25,243::task::1172::TaskManager.Task::(prepare) Task=`768cd67e-8b36-4e98-849a-5ebcbb03ca0c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112380981445', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-822::DEBUG::2012-12-09 12:22:25,243::task::588::TaskManager.Task::(_updateState) Task=`768cd67e-8b36-4e98-849a-5ebcbb03ca0c`::moving from state preparing -> state finished
>Thread-822::DEBUG::2012-12-09 12:22:25,244::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-822::DEBUG::2012-12-09 12:22:25,244::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-822::DEBUG::2012-12-09 12:22:25,244::task::978::TaskManager.Task::(_decref) Task=`768cd67e-8b36-4e98-849a-5ebcbb03ca0c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:26,151::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:26,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214809 s, 47.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:22:26,770::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:26,788::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000885136 s, 4.6 MB/s\n'; <rc> = 0
>Thread-824::DEBUG::2012-12-09 12:22:26,805::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-824::DEBUG::2012-12-09 12:22:26,806::task::588::TaskManager.Task::(_updateState) Task=`697c7739-4b70-4277-8e82-fe019f06c6c8`::moving from state init -> state preparing
>Thread-824::INFO::2012-12-09 12:22:26,807::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-824::INFO::2012-12-09 12:22:26,807::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-824::DEBUG::2012-12-09 12:22:26,808::task::1172::TaskManager.Task::(prepare) Task=`697c7739-4b70-4277-8e82-fe019f06c6c8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-824::DEBUG::2012-12-09 12:22:26,808::task::588::TaskManager.Task::(_updateState) Task=`697c7739-4b70-4277-8e82-fe019f06c6c8`::moving from state preparing -> state finished
>Thread-824::DEBUG::2012-12-09 12:22:26,808::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-824::DEBUG::2012-12-09 12:22:26,809::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-824::DEBUG::2012-12-09 12:22:26,809::task::978::TaskManager.Task::(_decref) Task=`697c7739-4b70-4277-8e82-fe019f06c6c8`::ref 0 aborting False
>Thread-825::DEBUG::2012-12-09 12:22:26,833::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-825::DEBUG::2012-12-09 12:22:26,833::task::588::TaskManager.Task::(_updateState) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::moving from state init -> state preparing
>Thread-825::INFO::2012-12-09 12:22:26,834::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-825::DEBUG::2012-12-09 12:22:26,834::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9fe0c86b-518e-4d2d-8a4d-0dfef319b818`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-825::DEBUG::2012-12-09 12:22:26,835::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-825::DEBUG::2012-12-09 12:22:26,835::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-825::DEBUG::2012-12-09 12:22:26,836::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9fe0c86b-518e-4d2d-8a4d-0dfef319b818`::Granted request
>Thread-825::DEBUG::2012-12-09 12:22:26,836::task::817::TaskManager.Task::(resourceAcquired) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-825::DEBUG::2012-12-09 12:22:26,837::task::978::TaskManager.Task::(_decref) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::ref 1 aborting False
>Thread-825::DEBUG::2012-12-09 12:22:26,838::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-825::DEBUG::2012-12-09 12:22:26,839::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-825::DEBUG::2012-12-09 12:22:26,948::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-825::WARNING::2012-12-09 12:22:26,950::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-825::DEBUG::2012-12-09 12:22:26,950::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-825::WARNING::2012-12-09 12:22:26,951::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-825::INFO::2012-12-09 12:22:26,951::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-825::DEBUG::2012-12-09 12:22:26,951::task::1172::TaskManager.Task::(prepare) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-825::DEBUG::2012-12-09 12:22:26,952::task::588::TaskManager.Task::(_updateState) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::moving from state preparing -> state finished
>Thread-825::DEBUG::2012-12-09 12:22:26,952::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-825::DEBUG::2012-12-09 12:22:26,952::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-825::DEBUG::2012-12-09 12:22:26,952::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-825::DEBUG::2012-12-09 12:22:26,953::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-825::DEBUG::2012-12-09 12:22:26,953::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-825::DEBUG::2012-12-09 12:22:26,953::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-825::DEBUG::2012-12-09 12:22:26,953::task::978::TaskManager.Task::(_decref) Task=`e9bbeac6-79e1-4dbd-9ecb-914309922613`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:28,204::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:28,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016321 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:30,251::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:30,297::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225867 s, 45.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:32,304::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:32,352::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:32,353::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:32,362::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266449 s, 38.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:32,458::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:32,460::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:32,461::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:32,461::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:34,369::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:34,408::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155961 s, 65.7 MB/s\n'; <rc> = 0
>Thread-830::DEBUG::2012-12-09 12:22:35,528::task::588::TaskManager.Task::(_updateState) Task=`8c6ae080-c524-4392-b58b-09096b0b9455`::moving from state init -> state preparing
>Thread-830::INFO::2012-12-09 12:22:35,529::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-830::INFO::2012-12-09 12:22:35,530::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0189127922058', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-830::DEBUG::2012-12-09 12:22:35,530::task::1172::TaskManager.Task::(prepare) Task=`8c6ae080-c524-4392-b58b-09096b0b9455`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0189127922058', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-830::DEBUG::2012-12-09 12:22:35,530::task::588::TaskManager.Task::(_updateState) Task=`8c6ae080-c524-4392-b58b-09096b0b9455`::moving from state preparing -> state finished
>Thread-830::DEBUG::2012-12-09 12:22:35,531::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-830::DEBUG::2012-12-09 12:22:35,531::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-830::DEBUG::2012-12-09 12:22:35,531::task::978::TaskManager.Task::(_decref) Task=`8c6ae080-c524-4392-b58b-09096b0b9455`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:36,415::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:36,465::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249309 s, 41.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:22:36,795::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:36,803::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000564077 s, 7.3 MB/s\n'; <rc> = 0
>Thread-832::DEBUG::2012-12-09 12:22:37,146::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-832::DEBUG::2012-12-09 12:22:37,146::task::588::TaskManager.Task::(_updateState) Task=`94e4176b-59f0-4c5e-be60-8b1711f477f7`::moving from state init -> state preparing
>Thread-832::INFO::2012-12-09 12:22:37,146::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-832::INFO::2012-12-09 12:22:37,147::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-832::DEBUG::2012-12-09 12:22:37,147::task::1172::TaskManager.Task::(prepare) Task=`94e4176b-59f0-4c5e-be60-8b1711f477f7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-832::DEBUG::2012-12-09 12:22:37,147::task::588::TaskManager.Task::(_updateState) Task=`94e4176b-59f0-4c5e-be60-8b1711f477f7`::moving from state preparing -> state finished
>Thread-832::DEBUG::2012-12-09 12:22:37,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-832::DEBUG::2012-12-09 12:22:37,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-832::DEBUG::2012-12-09 12:22:37,148::task::978::TaskManager.Task::(_decref) Task=`94e4176b-59f0-4c5e-be60-8b1711f477f7`::ref 0 aborting False
>Thread-833::DEBUG::2012-12-09 12:22:37,161::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-833::DEBUG::2012-12-09 12:22:37,161::task::588::TaskManager.Task::(_updateState) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::moving from state init -> state preparing
>Thread-833::INFO::2012-12-09 12:22:37,162::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-833::DEBUG::2012-12-09 12:22:37,162::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4683adc3-3e46-4a62-8ff6-28085deaf39f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-833::DEBUG::2012-12-09 12:22:37,162::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-833::DEBUG::2012-12-09 12:22:37,163::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-833::DEBUG::2012-12-09 12:22:37,163::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4683adc3-3e46-4a62-8ff6-28085deaf39f`::Granted request
>Thread-833::DEBUG::2012-12-09 12:22:37,163::task::817::TaskManager.Task::(resourceAcquired) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-833::DEBUG::2012-12-09 12:22:37,163::task::978::TaskManager.Task::(_decref) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::ref 1 aborting False
>Thread-833::DEBUG::2012-12-09 12:22:37,164::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-833::DEBUG::2012-12-09 12:22:37,164::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-833::DEBUG::2012-12-09 12:22:37,259::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-833::WARNING::2012-12-09 12:22:37,260::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-833::DEBUG::2012-12-09 12:22:37,260::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-833::WARNING::2012-12-09 12:22:37,261::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-833::INFO::2012-12-09 12:22:37,261::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-833::DEBUG::2012-12-09 12:22:37,261::task::1172::TaskManager.Task::(prepare) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-833::DEBUG::2012-12-09 12:22:37,262::task::588::TaskManager.Task::(_updateState) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::moving from state preparing -> state finished
>Thread-833::DEBUG::2012-12-09 12:22:37,262::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-833::DEBUG::2012-12-09 12:22:37,262::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-833::DEBUG::2012-12-09 12:22:37,262::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-833::DEBUG::2012-12-09 12:22:37,263::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-833::DEBUG::2012-12-09 12:22:37,263::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-833::DEBUG::2012-12-09 12:22:37,263::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-833::DEBUG::2012-12-09 12:22:37,263::task::978::TaskManager.Task::(_decref) Task=`0ffae91e-7dea-4b4e-a8a2-8934886eb3dc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:38,472::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:38,526::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268702 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:40,534::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:40,584::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269137 s, 38.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:42,462::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:42,463::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:42,571::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:42,572::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:42,572::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:42,573::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:42,591::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:42,636::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207644 s, 49.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:44,643::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:44,694::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268287 s, 38.2 MB/s\n'; <rc> = 0
>Thread-838::DEBUG::2012-12-09 12:22:45,769::task::588::TaskManager.Task::(_updateState) Task=`0d7643a6-6175-4b7b-862d-2ea91f6409a9`::moving from state init -> state preparing
>Thread-838::INFO::2012-12-09 12:22:45,770::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-838::INFO::2012-12-09 12:22:45,771::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00789594650269', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-838::DEBUG::2012-12-09 12:22:45,771::task::1172::TaskManager.Task::(prepare) Task=`0d7643a6-6175-4b7b-862d-2ea91f6409a9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00789594650269', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-838::DEBUG::2012-12-09 12:22:45,771::task::588::TaskManager.Task::(_updateState) Task=`0d7643a6-6175-4b7b-862d-2ea91f6409a9`::moving from state preparing -> state finished
>Thread-838::DEBUG::2012-12-09 12:22:45,771::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-838::DEBUG::2012-12-09 12:22:45,772::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-838::DEBUG::2012-12-09 12:22:45,772::task::978::TaskManager.Task::(_decref) Task=`0d7643a6-6175-4b7b-862d-2ea91f6409a9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:46,701::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:46,750::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240192 s, 42.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:22:46,806::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:46,819::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000946949 s, 4.3 MB/s\n'; <rc> = 0
>Thread-840::DEBUG::2012-12-09 12:22:47,355::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-840::DEBUG::2012-12-09 12:22:47,356::task::588::TaskManager.Task::(_updateState) Task=`268cbc5b-72a4-45ad-be2e-66ba2622e0d8`::moving from state init -> state preparing
>Thread-840::INFO::2012-12-09 12:22:47,356::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-840::INFO::2012-12-09 12:22:47,357::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-840::DEBUG::2012-12-09 12:22:47,357::task::1172::TaskManager.Task::(prepare) Task=`268cbc5b-72a4-45ad-be2e-66ba2622e0d8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-840::DEBUG::2012-12-09 12:22:47,358::task::588::TaskManager.Task::(_updateState) Task=`268cbc5b-72a4-45ad-be2e-66ba2622e0d8`::moving from state preparing -> state finished
>Thread-840::DEBUG::2012-12-09 12:22:47,358::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-840::DEBUG::2012-12-09 12:22:47,359::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-840::DEBUG::2012-12-09 12:22:47,359::task::978::TaskManager.Task::(_decref) Task=`268cbc5b-72a4-45ad-be2e-66ba2622e0d8`::ref 0 aborting False
>Thread-841::DEBUG::2012-12-09 12:22:47,382::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-841::DEBUG::2012-12-09 12:22:47,383::task::588::TaskManager.Task::(_updateState) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::moving from state init -> state preparing
>Thread-841::INFO::2012-12-09 12:22:47,384::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-841::DEBUG::2012-12-09 12:22:47,384::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`368c6bbb-e036-4c95-a02d-d99dbdf88520`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-841::DEBUG::2012-12-09 12:22:47,385::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-841::DEBUG::2012-12-09 12:22:47,385::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-841::DEBUG::2012-12-09 12:22:47,386::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`368c6bbb-e036-4c95-a02d-d99dbdf88520`::Granted request
>Thread-841::DEBUG::2012-12-09 12:22:47,386::task::817::TaskManager.Task::(resourceAcquired) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-841::DEBUG::2012-12-09 12:22:47,387::task::978::TaskManager.Task::(_decref) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::ref 1 aborting False
>Thread-841::DEBUG::2012-12-09 12:22:47,388::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-841::DEBUG::2012-12-09 12:22:47,389::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-841::DEBUG::2012-12-09 12:22:47,496::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-841::WARNING::2012-12-09 12:22:47,498::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-841::DEBUG::2012-12-09 12:22:47,498::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-841::WARNING::2012-12-09 12:22:47,498::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-841::INFO::2012-12-09 12:22:47,499::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-841::DEBUG::2012-12-09 12:22:47,500::task::1172::TaskManager.Task::(prepare) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-841::DEBUG::2012-12-09 12:22:47,500::task::588::TaskManager.Task::(_updateState) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::moving from state preparing -> state finished
>Thread-841::DEBUG::2012-12-09 12:22:47,501::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-841::DEBUG::2012-12-09 12:22:47,501::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-841::DEBUG::2012-12-09 12:22:47,501::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-841::DEBUG::2012-12-09 12:22:47,501::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-841::DEBUG::2012-12-09 12:22:47,501::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-841::DEBUG::2012-12-09 12:22:47,502::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-841::DEBUG::2012-12-09 12:22:47,502::task::978::TaskManager.Task::(_decref) Task=`c542aa1c-e076-4e47-b2ee-8e65faec77bb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:48,757::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:48,809::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255211 s, 40.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:50,816::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:50,855::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157804 s, 64.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:22:52,573::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:22:52,574::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:22:52,670::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:22:52,671::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:22:52,671::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:22:52,672::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:22:52,861::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:52,901::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016554 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:54,908::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:54,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017388 s, 58.9 MB/s\n'; <rc> = 0
>Thread-846::DEBUG::2012-12-09 12:22:56,175::task::588::TaskManager.Task::(_updateState) Task=`af566580-ee6a-4e05-9995-e013992fa2e1`::moving from state init -> state preparing
>Thread-846::INFO::2012-12-09 12:22:56,176::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-846::INFO::2012-12-09 12:22:56,177::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146429538727', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-846::DEBUG::2012-12-09 12:22:56,177::task::1172::TaskManager.Task::(prepare) Task=`af566580-ee6a-4e05-9995-e013992fa2e1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146429538727', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-846::DEBUG::2012-12-09 12:22:56,177::task::588::TaskManager.Task::(_updateState) Task=`af566580-ee6a-4e05-9995-e013992fa2e1`::moving from state preparing -> state finished
>Thread-846::DEBUG::2012-12-09 12:22:56,178::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-846::DEBUG::2012-12-09 12:22:56,178::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-846::DEBUG::2012-12-09 12:22:56,179::task::978::TaskManager.Task::(_decref) Task=`af566580-ee6a-4e05-9995-e013992fa2e1`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:22:56,823::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:22:56,831::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000414566 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:22:56,956::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:56,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157616 s, 65.0 MB/s\n'; <rc> = 0
>Thread-848::DEBUG::2012-12-09 12:22:57,662::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-848::DEBUG::2012-12-09 12:22:57,663::task::588::TaskManager.Task::(_updateState) Task=`3d01f99e-6cf7-4f74-9cde-3b63b1ff4927`::moving from state init -> state preparing
>Thread-848::INFO::2012-12-09 12:22:57,663::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-848::INFO::2012-12-09 12:22:57,664::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-848::DEBUG::2012-12-09 12:22:57,664::task::1172::TaskManager.Task::(prepare) Task=`3d01f99e-6cf7-4f74-9cde-3b63b1ff4927`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-848::DEBUG::2012-12-09 12:22:57,665::task::588::TaskManager.Task::(_updateState) Task=`3d01f99e-6cf7-4f74-9cde-3b63b1ff4927`::moving from state preparing -> state finished
>Thread-848::DEBUG::2012-12-09 12:22:57,665::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-848::DEBUG::2012-12-09 12:22:57,665::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-848::DEBUG::2012-12-09 12:22:57,666::task::978::TaskManager.Task::(_decref) Task=`3d01f99e-6cf7-4f74-9cde-3b63b1ff4927`::ref 0 aborting False
>Thread-849::DEBUG::2012-12-09 12:22:57,681::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-849::DEBUG::2012-12-09 12:22:57,681::task::588::TaskManager.Task::(_updateState) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::moving from state init -> state preparing
>Thread-849::INFO::2012-12-09 12:22:57,681::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-849::DEBUG::2012-12-09 12:22:57,682::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`335c49af-279d-4307-a26f-3ff27bc33e28`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-849::DEBUG::2012-12-09 12:22:57,682::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-849::DEBUG::2012-12-09 12:22:57,682::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-849::DEBUG::2012-12-09 12:22:57,683::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`335c49af-279d-4307-a26f-3ff27bc33e28`::Granted request
>Thread-849::DEBUG::2012-12-09 12:22:57,683::task::817::TaskManager.Task::(resourceAcquired) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-849::DEBUG::2012-12-09 12:22:57,683::task::978::TaskManager.Task::(_decref) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::ref 1 aborting False
>Thread-849::DEBUG::2012-12-09 12:22:57,684::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-849::DEBUG::2012-12-09 12:22:57,685::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-849::DEBUG::2012-12-09 12:22:57,783::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-849::WARNING::2012-12-09 12:22:57,784::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-849::DEBUG::2012-12-09 12:22:57,784::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-849::WARNING::2012-12-09 12:22:57,785::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-849::INFO::2012-12-09 12:22:57,785::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-849::DEBUG::2012-12-09 12:22:57,785::task::1172::TaskManager.Task::(prepare) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-849::DEBUG::2012-12-09 12:22:57,786::task::588::TaskManager.Task::(_updateState) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::moving from state preparing -> state finished
>Thread-849::DEBUG::2012-12-09 12:22:57,786::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-849::DEBUG::2012-12-09 12:22:57,787::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-849::DEBUG::2012-12-09 12:22:57,787::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-849::DEBUG::2012-12-09 12:22:57,787::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-849::DEBUG::2012-12-09 12:22:57,787::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-849::DEBUG::2012-12-09 12:22:57,788::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-849::DEBUG::2012-12-09 12:22:57,788::task::978::TaskManager.Task::(_decref) Task=`89cc6616-f5ea-42ba-b458-7d039af3abb5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:22:59,003::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:22:59,052::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203155 s, 50.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:01,059::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:01,099::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174678 s, 58.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:23:02,673::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:02,674::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:02,784::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:02,786::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:02,786::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:02,786::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:23:03,106::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:03,145::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165359 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:05,152::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:05,198::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021537 s, 47.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:06,834::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:06,846::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000551106 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:07,205::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:07,246::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166475 s, 61.5 MB/s\n'; <rc> = 0
>Thread-854::DEBUG::2012-12-09 12:23:08,617::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-854::DEBUG::2012-12-09 12:23:08,618::task::588::TaskManager.Task::(_updateState) Task=`92118b96-421c-42f5-be0a-8e4331d2a7d6`::moving from state init -> state preparing
>Thread-854::INFO::2012-12-09 12:23:08,618::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-854::INFO::2012-12-09 12:23:08,619::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-854::DEBUG::2012-12-09 12:23:08,619::task::1172::TaskManager.Task::(prepare) Task=`92118b96-421c-42f5-be0a-8e4331d2a7d6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-854::DEBUG::2012-12-09 12:23:08,619::task::588::TaskManager.Task::(_updateState) Task=`92118b96-421c-42f5-be0a-8e4331d2a7d6`::moving from state preparing -> state finished
>Thread-854::DEBUG::2012-12-09 12:23:08,620::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-854::DEBUG::2012-12-09 12:23:08,620::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-854::DEBUG::2012-12-09 12:23:08,620::task::978::TaskManager.Task::(_decref) Task=`92118b96-421c-42f5-be0a-8e4331d2a7d6`::ref 0 aborting False
>Thread-855::DEBUG::2012-12-09 12:23:08,635::task::588::TaskManager.Task::(_updateState) Task=`2ebbd436-dfa9-4bb2-b37b-c001066c4f66`::moving from state init -> state preparing
>Thread-855::INFO::2012-12-09 12:23:08,636::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-856::DEBUG::2012-12-09 12:23:08,636::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-855::INFO::2012-12-09 12:23:08,637::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124659538269', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-856::DEBUG::2012-12-09 12:23:08,638::task::588::TaskManager.Task::(_updateState) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::moving from state init -> state preparing
>Thread-855::DEBUG::2012-12-09 12:23:08,638::task::1172::TaskManager.Task::(prepare) Task=`2ebbd436-dfa9-4bb2-b37b-c001066c4f66`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124659538269', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-856::INFO::2012-12-09 12:23:08,638::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-855::DEBUG::2012-12-09 12:23:08,638::task::588::TaskManager.Task::(_updateState) Task=`2ebbd436-dfa9-4bb2-b37b-c001066c4f66`::moving from state preparing -> state finished
>Thread-856::DEBUG::2012-12-09 12:23:08,639::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`69dc629e-bdd8-4d2a-9615-69f6ffb98558`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-855::DEBUG::2012-12-09 12:23:08,640::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-856::DEBUG::2012-12-09 12:23:08,640::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-855::DEBUG::2012-12-09 12:23:08,640::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-856::DEBUG::2012-12-09 12:23:08,641::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-855::DEBUG::2012-12-09 12:23:08,642::task::978::TaskManager.Task::(_decref) Task=`2ebbd436-dfa9-4bb2-b37b-c001066c4f66`::ref 0 aborting False
>Thread-856::DEBUG::2012-12-09 12:23:08,643::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`69dc629e-bdd8-4d2a-9615-69f6ffb98558`::Granted request
>Thread-856::DEBUG::2012-12-09 12:23:08,649::task::817::TaskManager.Task::(resourceAcquired) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-856::DEBUG::2012-12-09 12:23:08,650::task::978::TaskManager.Task::(_decref) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::ref 1 aborting False
>Thread-856::DEBUG::2012-12-09 12:23:08,653::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-856::DEBUG::2012-12-09 12:23:08,654::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-856::DEBUG::2012-12-09 12:23:08,754::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-856::WARNING::2012-12-09 12:23:08,756::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-856::DEBUG::2012-12-09 12:23:08,757::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-856::WARNING::2012-12-09 12:23:08,758::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-856::INFO::2012-12-09 12:23:08,759::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-856::DEBUG::2012-12-09 12:23:08,759::task::1172::TaskManager.Task::(prepare) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-856::DEBUG::2012-12-09 12:23:08,760::task::588::TaskManager.Task::(_updateState) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::moving from state preparing -> state finished
>Thread-856::DEBUG::2012-12-09 12:23:08,760::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-856::DEBUG::2012-12-09 12:23:08,760::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-856::DEBUG::2012-12-09 12:23:08,761::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-856::DEBUG::2012-12-09 12:23:08,761::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-856::DEBUG::2012-12-09 12:23:08,761::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-856::DEBUG::2012-12-09 12:23:08,761::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-856::DEBUG::2012-12-09 12:23:08,762::task::978::TaskManager.Task::(_decref) Task=`1810fe27-94c8-44b4-b479-87a49a86e59c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:09,252::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:09,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170776 s, 60.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:11,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:11,342::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180335 s, 56.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:23:12,787::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:12,788::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:12,901::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:12,902::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:12,902::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:12,903::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:23:13,348::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:13,391::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159218 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:15,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:15,439::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174801 s, 58.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:16,849::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:16,861::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000422626 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:17,446::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:17,501::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270542 s, 37.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:19,508::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-862::DEBUG::2012-12-09 12:23:19,553::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-862::DEBUG::2012-12-09 12:23:19,554::task::588::TaskManager.Task::(_updateState) Task=`50665708-f4f1-420e-b225-919dce3eaeca`::moving from state init -> state preparing
>Thread-862::INFO::2012-12-09 12:23:19,555::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-862::INFO::2012-12-09 12:23:19,555::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-862::DEBUG::2012-12-09 12:23:19,555::task::1172::TaskManager.Task::(prepare) Task=`50665708-f4f1-420e-b225-919dce3eaeca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-862::DEBUG::2012-12-09 12:23:19,555::task::588::TaskManager.Task::(_updateState) Task=`50665708-f4f1-420e-b225-919dce3eaeca`::moving from state preparing -> state finished
>Thread-862::DEBUG::2012-12-09 12:23:19,556::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-862::DEBUG::2012-12-09 12:23:19,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-862::DEBUG::2012-12-09 12:23:19,556::task::978::TaskManager.Task::(_decref) Task=`50665708-f4f1-420e-b225-919dce3eaeca`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:19,569::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266422 s, 38.4 MB/s\n'; <rc> = 0
>Thread-863::DEBUG::2012-12-09 12:23:19,587::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-863::DEBUG::2012-12-09 12:23:19,588::task::588::TaskManager.Task::(_updateState) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::moving from state init -> state preparing
>Thread-863::INFO::2012-12-09 12:23:19,588::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-863::DEBUG::2012-12-09 12:23:19,589::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bf3f5ba7-d31f-4bfd-9dd7-fe1bc9d6af54`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-863::DEBUG::2012-12-09 12:23:19,589::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-863::DEBUG::2012-12-09 12:23:19,589::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-863::DEBUG::2012-12-09 12:23:19,589::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bf3f5ba7-d31f-4bfd-9dd7-fe1bc9d6af54`::Granted request
>Thread-863::DEBUG::2012-12-09 12:23:19,590::task::817::TaskManager.Task::(resourceAcquired) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-863::DEBUG::2012-12-09 12:23:19,590::task::978::TaskManager.Task::(_decref) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::ref 1 aborting False
>Thread-863::DEBUG::2012-12-09 12:23:19,591::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-863::DEBUG::2012-12-09 12:23:19,591::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-864::DEBUG::2012-12-09 12:23:19,609::task::588::TaskManager.Task::(_updateState) Task=`f7f624a4-2de1-4cc6-95c9-e15d5921a609`::moving from state init -> state preparing
>Thread-864::INFO::2012-12-09 12:23:19,609::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-864::INFO::2012-12-09 12:23:19,610::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127620697021', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-864::DEBUG::2012-12-09 12:23:19,610::task::1172::TaskManager.Task::(prepare) Task=`f7f624a4-2de1-4cc6-95c9-e15d5921a609`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127620697021', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-864::DEBUG::2012-12-09 12:23:19,610::task::588::TaskManager.Task::(_updateState) Task=`f7f624a4-2de1-4cc6-95c9-e15d5921a609`::moving from state preparing -> state finished
>Thread-864::DEBUG::2012-12-09 12:23:19,610::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-864::DEBUG::2012-12-09 12:23:19,610::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-864::DEBUG::2012-12-09 12:23:19,611::task::978::TaskManager.Task::(_decref) Task=`f7f624a4-2de1-4cc6-95c9-e15d5921a609`::ref 0 aborting False
>Thread-863::DEBUG::2012-12-09 12:23:19,695::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-863::WARNING::2012-12-09 12:23:19,697::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-863::DEBUG::2012-12-09 12:23:19,697::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-863::WARNING::2012-12-09 12:23:19,698::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-863::INFO::2012-12-09 12:23:19,699::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-863::DEBUG::2012-12-09 12:23:19,699::task::1172::TaskManager.Task::(prepare) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-863::DEBUG::2012-12-09 12:23:19,700::task::588::TaskManager.Task::(_updateState) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::moving from state preparing -> state finished
>Thread-863::DEBUG::2012-12-09 12:23:19,700::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-863::DEBUG::2012-12-09 12:23:19,701::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-863::DEBUG::2012-12-09 12:23:19,701::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-863::DEBUG::2012-12-09 12:23:19,701::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-863::DEBUG::2012-12-09 12:23:19,701::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-863::DEBUG::2012-12-09 12:23:19,701::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-863::DEBUG::2012-12-09 12:23:19,702::task::978::TaskManager.Task::(_decref) Task=`369a4fe3-b8f7-4236-8cf8-6f57a3900a72`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:21,577::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:21,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244846 s, 41.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:23:22,904::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:22,905::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:23,012::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:23,014::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:23,014::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:23,015::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:23:23,635::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:23,680::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188886 s, 54.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:25,687::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:25,730::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186584 s, 54.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:26,864::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:26,890::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000543306 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:27,737::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:27,780::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01721 s, 59.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:29,786::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:29,828::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172118 s, 59.5 MB/s\n'; <rc> = 0
>Thread-870::DEBUG::2012-12-09 12:23:30,249::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-870::DEBUG::2012-12-09 12:23:30,249::task::588::TaskManager.Task::(_updateState) Task=`7d74e2cf-4bc5-4c1f-96d5-185d9c889191`::moving from state init -> state preparing
>Thread-870::INFO::2012-12-09 12:23:30,249::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-870::INFO::2012-12-09 12:23:30,250::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-870::DEBUG::2012-12-09 12:23:30,250::task::1172::TaskManager.Task::(prepare) Task=`7d74e2cf-4bc5-4c1f-96d5-185d9c889191`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-870::DEBUG::2012-12-09 12:23:30,250::task::588::TaskManager.Task::(_updateState) Task=`7d74e2cf-4bc5-4c1f-96d5-185d9c889191`::moving from state preparing -> state finished
>Thread-870::DEBUG::2012-12-09 12:23:30,250::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-870::DEBUG::2012-12-09 12:23:30,250::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-870::DEBUG::2012-12-09 12:23:30,251::task::978::TaskManager.Task::(_decref) Task=`7d74e2cf-4bc5-4c1f-96d5-185d9c889191`::ref 0 aborting False
>Thread-871::DEBUG::2012-12-09 12:23:30,267::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-871::DEBUG::2012-12-09 12:23:30,268::task::588::TaskManager.Task::(_updateState) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::moving from state init -> state preparing
>Thread-871::INFO::2012-12-09 12:23:30,268::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-871::DEBUG::2012-12-09 12:23:30,269::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d80637f-ed7a-4622-98c7-2cd45e835588`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-871::DEBUG::2012-12-09 12:23:30,269::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-871::DEBUG::2012-12-09 12:23:30,269::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-871::DEBUG::2012-12-09 12:23:30,269::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d80637f-ed7a-4622-98c7-2cd45e835588`::Granted request
>Thread-871::DEBUG::2012-12-09 12:23:30,270::task::817::TaskManager.Task::(resourceAcquired) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-871::DEBUG::2012-12-09 12:23:30,270::task::978::TaskManager.Task::(_decref) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::ref 1 aborting False
>Thread-871::DEBUG::2012-12-09 12:23:30,271::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-871::DEBUG::2012-12-09 12:23:30,271::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-871::DEBUG::2012-12-09 12:23:30,371::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-871::WARNING::2012-12-09 12:23:30,372::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-871::DEBUG::2012-12-09 12:23:30,373::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-871::WARNING::2012-12-09 12:23:30,373::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-871::INFO::2012-12-09 12:23:30,374::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-871::DEBUG::2012-12-09 12:23:30,374::task::1172::TaskManager.Task::(prepare) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-871::DEBUG::2012-12-09 12:23:30,374::task::588::TaskManager.Task::(_updateState) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::moving from state preparing -> state finished
>Thread-871::DEBUG::2012-12-09 12:23:30,374::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-871::DEBUG::2012-12-09 12:23:30,375::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-871::DEBUG::2012-12-09 12:23:30,375::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-871::DEBUG::2012-12-09 12:23:30,375::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-871::DEBUG::2012-12-09 12:23:30,375::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-871::DEBUG::2012-12-09 12:23:30,376::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-871::DEBUG::2012-12-09 12:23:30,376::task::978::TaskManager.Task::(_decref) Task=`0d51080a-94c9-48ec-a2e3-e8a2eba889e9`::ref 0 aborting False
>Thread-872::DEBUG::2012-12-09 12:23:31,589::task::588::TaskManager.Task::(_updateState) Task=`c3875e35-c6a7-4659-a0f8-461db9eacc5d`::moving from state init -> state preparing
>Thread-872::INFO::2012-12-09 12:23:31,589::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-872::INFO::2012-12-09 12:23:31,589::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263750553131', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-872::DEBUG::2012-12-09 12:23:31,590::task::1172::TaskManager.Task::(prepare) Task=`c3875e35-c6a7-4659-a0f8-461db9eacc5d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263750553131', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-872::DEBUG::2012-12-09 12:23:31,590::task::588::TaskManager.Task::(_updateState) Task=`c3875e35-c6a7-4659-a0f8-461db9eacc5d`::moving from state preparing -> state finished
>Thread-872::DEBUG::2012-12-09 12:23:31,590::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-872::DEBUG::2012-12-09 12:23:31,590::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-872::DEBUG::2012-12-09 12:23:31,590::task::978::TaskManager.Task::(_decref) Task=`c3875e35-c6a7-4659-a0f8-461db9eacc5d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:31,835::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:31,878::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175059 s, 58.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:23:33,015::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:33,017::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:33,118::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:33,120::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:33,120::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:33,120::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:23:33,885::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:33,929::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195822 s, 52.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:35,936::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:35,977::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016869 s, 60.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:36,893::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:36,903::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00033786 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:37,984::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:38,030::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201841 s, 50.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:40,037::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:40,084::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197726 s, 51.8 MB/s\n'; <rc> = 0
>Thread-877::DEBUG::2012-12-09 12:23:40,666::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-877::DEBUG::2012-12-09 12:23:40,667::task::588::TaskManager.Task::(_updateState) Task=`f459d84e-27a3-401d-a9a1-8ac598c0e151`::moving from state init -> state preparing
>Thread-877::INFO::2012-12-09 12:23:40,667::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-877::INFO::2012-12-09 12:23:40,667::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-877::DEBUG::2012-12-09 12:23:40,667::task::1172::TaskManager.Task::(prepare) Task=`f459d84e-27a3-401d-a9a1-8ac598c0e151`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-877::DEBUG::2012-12-09 12:23:40,668::task::588::TaskManager.Task::(_updateState) Task=`f459d84e-27a3-401d-a9a1-8ac598c0e151`::moving from state preparing -> state finished
>Thread-877::DEBUG::2012-12-09 12:23:40,668::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-877::DEBUG::2012-12-09 12:23:40,669::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-877::DEBUG::2012-12-09 12:23:40,669::task::978::TaskManager.Task::(_decref) Task=`f459d84e-27a3-401d-a9a1-8ac598c0e151`::ref 0 aborting False
>Thread-878::DEBUG::2012-12-09 12:23:40,686::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-878::DEBUG::2012-12-09 12:23:40,686::task::588::TaskManager.Task::(_updateState) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::moving from state init -> state preparing
>Thread-878::INFO::2012-12-09 12:23:40,687::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-878::DEBUG::2012-12-09 12:23:40,688::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`68a0a5bb-a010-46eb-93ee-7c3d9dd37769`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-878::DEBUG::2012-12-09 12:23:40,688::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-878::DEBUG::2012-12-09 12:23:40,689::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-878::DEBUG::2012-12-09 12:23:40,689::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`68a0a5bb-a010-46eb-93ee-7c3d9dd37769`::Granted request
>Thread-878::DEBUG::2012-12-09 12:23:40,690::task::817::TaskManager.Task::(resourceAcquired) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-878::DEBUG::2012-12-09 12:23:40,690::task::978::TaskManager.Task::(_decref) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::ref 1 aborting False
>Thread-878::DEBUG::2012-12-09 12:23:40,691::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-878::DEBUG::2012-12-09 12:23:40,691::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-878::DEBUG::2012-12-09 12:23:40,786::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-878::WARNING::2012-12-09 12:23:40,787::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-878::DEBUG::2012-12-09 12:23:40,788::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-878::WARNING::2012-12-09 12:23:40,788::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-878::INFO::2012-12-09 12:23:40,788::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-878::DEBUG::2012-12-09 12:23:40,789::task::1172::TaskManager.Task::(prepare) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-878::DEBUG::2012-12-09 12:23:40,789::task::588::TaskManager.Task::(_updateState) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::moving from state preparing -> state finished
>Thread-878::DEBUG::2012-12-09 12:23:40,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-878::DEBUG::2012-12-09 12:23:40,790::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-878::DEBUG::2012-12-09 12:23:40,790::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-878::DEBUG::2012-12-09 12:23:40,790::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-878::DEBUG::2012-12-09 12:23:40,790::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-878::DEBUG::2012-12-09 12:23:40,791::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-878::DEBUG::2012-12-09 12:23:40,791::task::978::TaskManager.Task::(_decref) Task=`0b38b8e4-52c4-4f16-b54e-f8227cab23c8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:42,091::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:42,133::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187459 s, 54.6 MB/s\n'; <rc> = 0
>Thread-880::DEBUG::2012-12-09 12:23:42,833::task::588::TaskManager.Task::(_updateState) Task=`fd044e6e-9413-485a-b4d8-c7cc47826dc9`::moving from state init -> state preparing
>Thread-880::INFO::2012-12-09 12:23:42,834::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-880::INFO::2012-12-09 12:23:42,834::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101912021637', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-880::DEBUG::2012-12-09 12:23:42,834::task::1172::TaskManager.Task::(prepare) Task=`fd044e6e-9413-485a-b4d8-c7cc47826dc9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101912021637', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-880::DEBUG::2012-12-09 12:23:42,835::task::588::TaskManager.Task::(_updateState) Task=`fd044e6e-9413-485a-b4d8-c7cc47826dc9`::moving from state preparing -> state finished
>Thread-880::DEBUG::2012-12-09 12:23:42,835::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-880::DEBUG::2012-12-09 12:23:42,835::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-880::DEBUG::2012-12-09 12:23:42,835::task::978::TaskManager.Task::(_decref) Task=`fd044e6e-9413-485a-b4d8-c7cc47826dc9`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:23:43,121::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:43,122::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:43,222::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:43,223::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:43,223::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:43,224::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:23:44,139::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:44,187::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247237 s, 41.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:46,194::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:46,236::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185384 s, 55.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:46,906::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:46,921::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000700144 s, 5.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:48,242::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:48,284::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171061 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:50,290::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:50,329::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161423 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:52,336::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-886::DEBUG::2012-12-09 12:23:52,343::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-886::DEBUG::2012-12-09 12:23:52,346::task::588::TaskManager.Task::(_updateState) Task=`c0e49d15-a894-4346-8943-be437c38fe6d`::moving from state init -> state preparing
>Thread-886::INFO::2012-12-09 12:23:52,346::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-886::INFO::2012-12-09 12:23:52,348::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-886::DEBUG::2012-12-09 12:23:52,348::task::1172::TaskManager.Task::(prepare) Task=`c0e49d15-a894-4346-8943-be437c38fe6d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-886::DEBUG::2012-12-09 12:23:52,349::task::588::TaskManager.Task::(_updateState) Task=`c0e49d15-a894-4346-8943-be437c38fe6d`::moving from state preparing -> state finished
>Thread-886::DEBUG::2012-12-09 12:23:52,349::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-886::DEBUG::2012-12-09 12:23:52,350::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-886::DEBUG::2012-12-09 12:23:52,351::task::978::TaskManager.Task::(_decref) Task=`c0e49d15-a894-4346-8943-be437c38fe6d`::ref 0 aborting False
>Thread-887::DEBUG::2012-12-09 12:23:52,379::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-887::DEBUG::2012-12-09 12:23:52,383::task::588::TaskManager.Task::(_updateState) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::moving from state init -> state preparing
>Thread-887::INFO::2012-12-09 12:23:52,384::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-887::DEBUG::2012-12-09 12:23:52,386::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8de8df0f-c47b-4a96-b607-84201c7ed913`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-887::DEBUG::2012-12-09 12:23:52,387::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-887::DEBUG::2012-12-09 12:23:52,389::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-887::DEBUG::2012-12-09 12:23:52,391::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8de8df0f-c47b-4a96-b607-84201c7ed913`::Granted request
>Thread-887::DEBUG::2012-12-09 12:23:52,392::task::817::TaskManager.Task::(resourceAcquired) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-887::DEBUG::2012-12-09 12:23:52,394::task::978::TaskManager.Task::(_decref) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::ref 1 aborting False
>Thread-887::DEBUG::2012-12-09 12:23:52,397::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-887::DEBUG::2012-12-09 12:23:52,399::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:52,420::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277042 s, 37.0 MB/s\n'; <rc> = 0
>Thread-887::DEBUG::2012-12-09 12:23:52,502::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-887::WARNING::2012-12-09 12:23:52,504::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-887::DEBUG::2012-12-09 12:23:52,504::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-887::WARNING::2012-12-09 12:23:52,505::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-887::INFO::2012-12-09 12:23:52,506::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-887::DEBUG::2012-12-09 12:23:52,506::task::1172::TaskManager.Task::(prepare) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-887::DEBUG::2012-12-09 12:23:52,507::task::588::TaskManager.Task::(_updateState) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::moving from state preparing -> state finished
>Thread-887::DEBUG::2012-12-09 12:23:52,507::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-887::DEBUG::2012-12-09 12:23:52,508::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-887::DEBUG::2012-12-09 12:23:52,508::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-887::DEBUG::2012-12-09 12:23:52,509::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-887::DEBUG::2012-12-09 12:23:52,509::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-887::DEBUG::2012-12-09 12:23:52,510::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-887::DEBUG::2012-12-09 12:23:52,510::task::978::TaskManager.Task::(_decref) Task=`a4d358c1-47a3-4511-876d-84886d0230a2`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:23:53,225::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:23:53,226::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:23:53,327::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:23:53,329::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:23:53,329::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:23:53,330::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-888::DEBUG::2012-12-09 12:23:53,350::task::588::TaskManager.Task::(_updateState) Task=`f616c3e1-e7bf-487b-94ef-e3099dee526f`::moving from state init -> state preparing
>Thread-888::INFO::2012-12-09 12:23:53,350::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-888::INFO::2012-12-09 12:23:53,351::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154371261597', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-888::DEBUG::2012-12-09 12:23:53,351::task::1172::TaskManager.Task::(prepare) Task=`f616c3e1-e7bf-487b-94ef-e3099dee526f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154371261597', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-888::DEBUG::2012-12-09 12:23:53,351::task::588::TaskManager.Task::(_updateState) Task=`f616c3e1-e7bf-487b-94ef-e3099dee526f`::moving from state preparing -> state finished
>Thread-888::DEBUG::2012-12-09 12:23:53,351::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-888::DEBUG::2012-12-09 12:23:53,351::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-888::DEBUG::2012-12-09 12:23:53,352::task::978::TaskManager.Task::(_decref) Task=`f616c3e1-e7bf-487b-94ef-e3099dee526f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:23:54,426::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:54,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173086 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:56,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:56,520::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200352 s, 51.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:23:56,924::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:23:56,936::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000714068 s, 5.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:23:58,527::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:23:58,574::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193226 s, 53.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:00,581::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:00,627::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201481 s, 50.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:02,634::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:02,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017083 s, 59.9 MB/s\n'; <rc> = 0
>Thread-894::DEBUG::2012-12-09 12:24:02,853::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-894::DEBUG::2012-12-09 12:24:02,854::task::588::TaskManager.Task::(_updateState) Task=`c6c7b824-ad10-45d8-90d3-72c8959b733c`::moving from state init -> state preparing
>Thread-894::INFO::2012-12-09 12:24:02,854::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-894::INFO::2012-12-09 12:24:02,855::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-894::DEBUG::2012-12-09 12:24:02,855::task::1172::TaskManager.Task::(prepare) Task=`c6c7b824-ad10-45d8-90d3-72c8959b733c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-894::DEBUG::2012-12-09 12:24:02,856::task::588::TaskManager.Task::(_updateState) Task=`c6c7b824-ad10-45d8-90d3-72c8959b733c`::moving from state preparing -> state finished
>Thread-894::DEBUG::2012-12-09 12:24:02,856::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-894::DEBUG::2012-12-09 12:24:02,857::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-894::DEBUG::2012-12-09 12:24:02,857::task::978::TaskManager.Task::(_decref) Task=`c6c7b824-ad10-45d8-90d3-72c8959b733c`::ref 0 aborting False
>Thread-895::DEBUG::2012-12-09 12:24:02,876::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-895::DEBUG::2012-12-09 12:24:02,876::task::588::TaskManager.Task::(_updateState) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::moving from state init -> state preparing
>Thread-895::INFO::2012-12-09 12:24:02,877::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-895::DEBUG::2012-12-09 12:24:02,878::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`827fcf15-0ec3-4bbf-a62a-c67f061c96ac`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-895::DEBUG::2012-12-09 12:24:02,878::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-895::DEBUG::2012-12-09 12:24:02,879::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-895::DEBUG::2012-12-09 12:24:02,879::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`827fcf15-0ec3-4bbf-a62a-c67f061c96ac`::Granted request
>Thread-895::DEBUG::2012-12-09 12:24:02,880::task::817::TaskManager.Task::(resourceAcquired) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-895::DEBUG::2012-12-09 12:24:02,880::task::978::TaskManager.Task::(_decref) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::ref 1 aborting False
>Thread-895::DEBUG::2012-12-09 12:24:02,881::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-895::DEBUG::2012-12-09 12:24:02,881::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-895::DEBUG::2012-12-09 12:24:02,983::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-895::WARNING::2012-12-09 12:24:02,984::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-895::DEBUG::2012-12-09 12:24:02,985::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-895::WARNING::2012-12-09 12:24:02,985::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-895::INFO::2012-12-09 12:24:02,985::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-895::DEBUG::2012-12-09 12:24:02,986::task::1172::TaskManager.Task::(prepare) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-895::DEBUG::2012-12-09 12:24:02,986::task::588::TaskManager.Task::(_updateState) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::moving from state preparing -> state finished
>Thread-895::DEBUG::2012-12-09 12:24:02,986::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-895::DEBUG::2012-12-09 12:24:02,986::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-895::DEBUG::2012-12-09 12:24:02,986::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-895::DEBUG::2012-12-09 12:24:02,987::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-895::DEBUG::2012-12-09 12:24:02,987::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-895::DEBUG::2012-12-09 12:24:02,987::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-895::DEBUG::2012-12-09 12:24:02,987::task::978::TaskManager.Task::(_decref) Task=`cca214d0-6e17-4690-9b2f-fa503eb700a6`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:24:03,331::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:03,332::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:24:03,438::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:03,439::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:03,440::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:03,440::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-896::DEBUG::2012-12-09 12:24:03,688::task::588::TaskManager.Task::(_updateState) Task=`4467043e-b06a-42da-b9a5-cf4945b056f4`::moving from state init -> state preparing
>Thread-896::INFO::2012-12-09 12:24:03,689::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-896::INFO::2012-12-09 12:24:03,689::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122449398041', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-896::DEBUG::2012-12-09 12:24:03,689::task::1172::TaskManager.Task::(prepare) Task=`4467043e-b06a-42da-b9a5-cf4945b056f4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122449398041', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-896::DEBUG::2012-12-09 12:24:03,690::task::588::TaskManager.Task::(_updateState) Task=`4467043e-b06a-42da-b9a5-cf4945b056f4`::moving from state preparing -> state finished
>Thread-896::DEBUG::2012-12-09 12:24:03,690::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-896::DEBUG::2012-12-09 12:24:03,690::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-896::DEBUG::2012-12-09 12:24:03,691::task::978::TaskManager.Task::(_decref) Task=`4467043e-b06a-42da-b9a5-cf4945b056f4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:24:04,683::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:04,730::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214931 s, 47.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:06,737::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:06,778::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179333 s, 57.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:06,939::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:24:06,940::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:24:06,940::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:06,962::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000756384 s, 2.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:06,962::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000756384 s, 2.7 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:24:06,963::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:24:06,963::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:24:06,963::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:24:06,963::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:06,964::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:06,964::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:06,964::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:06,964::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:06,965::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:07,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:07,063::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:24:07,063::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:24:07,064::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:24:07,064::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:24:07,064::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:24:07,064::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:07,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:07,168::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:07,176::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000383265 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:08,785::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:08,833::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197184 s, 51.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:10,840::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:10,885::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202593 s, 50.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:12,892::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:12,934::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182528 s, 56.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:13,441::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:24:13,442::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:13,442::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:24:13,568::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:13,569::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:13,569::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:13,569::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:24:14,941::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:14,985::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173181 s, 59.1 MB/s\n'; <rc> = 0
>Thread-902::DEBUG::2012-12-09 12:24:15,472::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-902::DEBUG::2012-12-09 12:24:15,472::task::588::TaskManager.Task::(_updateState) Task=`24d93326-ece1-4708-8911-7dc2c44dafa5`::moving from state init -> state preparing
>Thread-902::INFO::2012-12-09 12:24:15,472::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-902::INFO::2012-12-09 12:24:15,473::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-902::DEBUG::2012-12-09 12:24:15,473::task::1172::TaskManager.Task::(prepare) Task=`24d93326-ece1-4708-8911-7dc2c44dafa5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-902::DEBUG::2012-12-09 12:24:15,473::task::588::TaskManager.Task::(_updateState) Task=`24d93326-ece1-4708-8911-7dc2c44dafa5`::moving from state preparing -> state finished
>Thread-902::DEBUG::2012-12-09 12:24:15,473::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-902::DEBUG::2012-12-09 12:24:15,474::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-902::DEBUG::2012-12-09 12:24:15,474::task::978::TaskManager.Task::(_decref) Task=`24d93326-ece1-4708-8911-7dc2c44dafa5`::ref 0 aborting False
>Thread-903::DEBUG::2012-12-09 12:24:15,484::task::588::TaskManager.Task::(_updateState) Task=`8f5cab8d-5755-422e-ac8a-b6c920ce215e`::moving from state init -> state preparing
>Thread-903::INFO::2012-12-09 12:24:15,485::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-903::INFO::2012-12-09 12:24:15,485::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00859999656677', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-903::DEBUG::2012-12-09 12:24:15,485::task::1172::TaskManager.Task::(prepare) Task=`8f5cab8d-5755-422e-ac8a-b6c920ce215e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00859999656677', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-903::DEBUG::2012-12-09 12:24:15,485::task::588::TaskManager.Task::(_updateState) Task=`8f5cab8d-5755-422e-ac8a-b6c920ce215e`::moving from state preparing -> state finished
>Thread-903::DEBUG::2012-12-09 12:24:15,486::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-903::DEBUG::2012-12-09 12:24:15,486::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-903::DEBUG::2012-12-09 12:24:15,486::task::978::TaskManager.Task::(_decref) Task=`8f5cab8d-5755-422e-ac8a-b6c920ce215e`::ref 0 aborting False
>Thread-904::DEBUG::2012-12-09 12:24:15,490::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-904::DEBUG::2012-12-09 12:24:15,490::task::588::TaskManager.Task::(_updateState) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::moving from state init -> state preparing
>Thread-904::INFO::2012-12-09 12:24:15,491::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-904::DEBUG::2012-12-09 12:24:15,491::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e8d5b099-1df2-436e-8f2a-49c4882ff30c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-904::DEBUG::2012-12-09 12:24:15,492::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-904::DEBUG::2012-12-09 12:24:15,492::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-904::DEBUG::2012-12-09 12:24:15,492::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e8d5b099-1df2-436e-8f2a-49c4882ff30c`::Granted request
>Thread-904::DEBUG::2012-12-09 12:24:15,493::task::817::TaskManager.Task::(resourceAcquired) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-904::DEBUG::2012-12-09 12:24:15,493::task::978::TaskManager.Task::(_decref) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::ref 1 aborting False
>Thread-904::DEBUG::2012-12-09 12:24:15,494::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-904::DEBUG::2012-12-09 12:24:15,495::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-904::DEBUG::2012-12-09 12:24:15,599::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-904::WARNING::2012-12-09 12:24:15,600::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-904::DEBUG::2012-12-09 12:24:15,600::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-904::WARNING::2012-12-09 12:24:15,600::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-904::INFO::2012-12-09 12:24:15,601::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-904::DEBUG::2012-12-09 12:24:15,601::task::1172::TaskManager.Task::(prepare) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-904::DEBUG::2012-12-09 12:24:15,601::task::588::TaskManager.Task::(_updateState) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::moving from state preparing -> state finished
>Thread-904::DEBUG::2012-12-09 12:24:15,601::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-904::DEBUG::2012-12-09 12:24:15,601::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-904::DEBUG::2012-12-09 12:24:15,602::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-904::DEBUG::2012-12-09 12:24:15,602::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-904::DEBUG::2012-12-09 12:24:15,602::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-904::DEBUG::2012-12-09 12:24:15,602::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-904::DEBUG::2012-12-09 12:24:15,603::task::978::TaskManager.Task::(_decref) Task=`3fc2c5e7-f346-41f3-88b8-8f6e5dc2ed15`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:24:16,992::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:17,039::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210267 s, 48.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:17,181::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:17,196::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000874876 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:19,046::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:19,089::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171279 s, 59.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:21,096::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:21,141::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188905 s, 54.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:23,147::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:23,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179216 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:23,570::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:23,571::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:24:23,672::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:23,673::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:23,673::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:23,674::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:24:25,197::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:25,242::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201002 s, 50.9 MB/s\n'; <rc> = 0
>Thread-910::DEBUG::2012-12-09 12:24:25,679::task::588::TaskManager.Task::(_updateState) Task=`8aea9094-d503-4472-95ce-3b2f13c8c437`::moving from state init -> state preparing
>Thread-910::INFO::2012-12-09 12:24:25,679::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-910::INFO::2012-12-09 12:24:25,679::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151388645172', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-910::DEBUG::2012-12-09 12:24:25,680::task::1172::TaskManager.Task::(prepare) Task=`8aea9094-d503-4472-95ce-3b2f13c8c437`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151388645172', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-910::DEBUG::2012-12-09 12:24:25,680::task::588::TaskManager.Task::(_updateState) Task=`8aea9094-d503-4472-95ce-3b2f13c8c437`::moving from state preparing -> state finished
>Thread-910::DEBUG::2012-12-09 12:24:25,680::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-910::DEBUG::2012-12-09 12:24:25,680::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-910::DEBUG::2012-12-09 12:24:25,681::task::978::TaskManager.Task::(_decref) Task=`8aea9094-d503-4472-95ce-3b2f13c8c437`::ref 0 aborting False
>Thread-912::DEBUG::2012-12-09 12:24:26,114::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-912::DEBUG::2012-12-09 12:24:26,116::task::588::TaskManager.Task::(_updateState) Task=`3cf75394-ae60-4ae3-a70b-fb488d56ba52`::moving from state init -> state preparing
>Thread-912::INFO::2012-12-09 12:24:26,117::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-912::INFO::2012-12-09 12:24:26,118::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-912::DEBUG::2012-12-09 12:24:26,119::task::1172::TaskManager.Task::(prepare) Task=`3cf75394-ae60-4ae3-a70b-fb488d56ba52`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-912::DEBUG::2012-12-09 12:24:26,120::task::588::TaskManager.Task::(_updateState) Task=`3cf75394-ae60-4ae3-a70b-fb488d56ba52`::moving from state preparing -> state finished
>Thread-912::DEBUG::2012-12-09 12:24:26,121::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-912::DEBUG::2012-12-09 12:24:26,121::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-912::DEBUG::2012-12-09 12:24:26,122::task::978::TaskManager.Task::(_decref) Task=`3cf75394-ae60-4ae3-a70b-fb488d56ba52`::ref 0 aborting False
>Thread-913::DEBUG::2012-12-09 12:24:26,142::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-913::DEBUG::2012-12-09 12:24:26,143::task::588::TaskManager.Task::(_updateState) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::moving from state init -> state preparing
>Thread-913::INFO::2012-12-09 12:24:26,144::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-913::DEBUG::2012-12-09 12:24:26,144::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b50462ad-533c-496d-acc8-1230b37d32e6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-913::DEBUG::2012-12-09 12:24:26,145::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-913::DEBUG::2012-12-09 12:24:26,145::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-913::DEBUG::2012-12-09 12:24:26,145::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b50462ad-533c-496d-acc8-1230b37d32e6`::Granted request
>Thread-913::DEBUG::2012-12-09 12:24:26,146::task::817::TaskManager.Task::(resourceAcquired) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-913::DEBUG::2012-12-09 12:24:26,146::task::978::TaskManager.Task::(_decref) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::ref 1 aborting False
>Thread-913::DEBUG::2012-12-09 12:24:26,148::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-913::DEBUG::2012-12-09 12:24:26,148::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-913::DEBUG::2012-12-09 12:24:26,245::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-913::WARNING::2012-12-09 12:24:26,246::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-913::DEBUG::2012-12-09 12:24:26,246::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-913::WARNING::2012-12-09 12:24:26,247::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-913::INFO::2012-12-09 12:24:26,247::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-913::DEBUG::2012-12-09 12:24:26,247::task::1172::TaskManager.Task::(prepare) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-913::DEBUG::2012-12-09 12:24:26,247::task::588::TaskManager.Task::(_updateState) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::moving from state preparing -> state finished
>Thread-913::DEBUG::2012-12-09 12:24:26,248::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-913::DEBUG::2012-12-09 12:24:26,248::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-913::DEBUG::2012-12-09 12:24:26,248::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-913::DEBUG::2012-12-09 12:24:26,248::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-913::DEBUG::2012-12-09 12:24:26,248::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-913::DEBUG::2012-12-09 12:24:26,249::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-913::DEBUG::2012-12-09 12:24:26,249::task::978::TaskManager.Task::(_decref) Task=`984d7da7-04b0-470f-8f4e-6a49a458122d`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:24:27,201::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:27,212::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000738511 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:27,249::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:27,287::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161214 s, 63.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:29,294::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:29,334::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017131 s, 59.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:31,342::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:31,383::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176518 s, 58.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:33,391::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:33,436::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199466 s, 51.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:33,675::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:33,676::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:24:33,781::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:33,783::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:33,783::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:33,783::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:24:35,443::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:35,485::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168773 s, 60.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:37,218::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:37,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000419093 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:37,492::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:37,534::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170077 s, 60.2 MB/s\n'; <rc> = 0
>Thread-918::DEBUG::2012-12-09 12:24:37,746::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-918::DEBUG::2012-12-09 12:24:37,746::task::588::TaskManager.Task::(_updateState) Task=`b85f75f5-d5ba-4dcc-b830-0dad9c23c1da`::moving from state init -> state preparing
>Thread-918::INFO::2012-12-09 12:24:37,746::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-918::INFO::2012-12-09 12:24:37,747::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-918::DEBUG::2012-12-09 12:24:37,747::task::1172::TaskManager.Task::(prepare) Task=`b85f75f5-d5ba-4dcc-b830-0dad9c23c1da`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-918::DEBUG::2012-12-09 12:24:37,747::task::588::TaskManager.Task::(_updateState) Task=`b85f75f5-d5ba-4dcc-b830-0dad9c23c1da`::moving from state preparing -> state finished
>Thread-918::DEBUG::2012-12-09 12:24:37,747::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-918::DEBUG::2012-12-09 12:24:37,747::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-918::DEBUG::2012-12-09 12:24:37,748::task::978::TaskManager.Task::(_decref) Task=`b85f75f5-d5ba-4dcc-b830-0dad9c23c1da`::ref 0 aborting False
>Thread-919::DEBUG::2012-12-09 12:24:37,769::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-919::DEBUG::2012-12-09 12:24:37,770::task::588::TaskManager.Task::(_updateState) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::moving from state init -> state preparing
>Thread-919::INFO::2012-12-09 12:24:37,770::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-919::DEBUG::2012-12-09 12:24:37,771::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c32b169c-3602-4500-856b-92139342a9ba`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-919::DEBUG::2012-12-09 12:24:37,771::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-919::DEBUG::2012-12-09 12:24:37,771::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-919::DEBUG::2012-12-09 12:24:37,771::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c32b169c-3602-4500-856b-92139342a9ba`::Granted request
>Thread-919::DEBUG::2012-12-09 12:24:37,772::task::817::TaskManager.Task::(resourceAcquired) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-919::DEBUG::2012-12-09 12:24:37,772::task::978::TaskManager.Task::(_decref) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::ref 1 aborting False
>Thread-919::DEBUG::2012-12-09 12:24:37,772::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-919::DEBUG::2012-12-09 12:24:37,773::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-920::DEBUG::2012-12-09 12:24:37,809::task::588::TaskManager.Task::(_updateState) Task=`fc546550-5964-4582-aee6-8eb0dbee6cbd`::moving from state init -> state preparing
>Thread-920::INFO::2012-12-09 12:24:37,809::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-920::INFO::2012-12-09 12:24:37,810::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.010617017746', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-920::DEBUG::2012-12-09 12:24:37,810::task::1172::TaskManager.Task::(prepare) Task=`fc546550-5964-4582-aee6-8eb0dbee6cbd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.010617017746', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-920::DEBUG::2012-12-09 12:24:37,811::task::588::TaskManager.Task::(_updateState) Task=`fc546550-5964-4582-aee6-8eb0dbee6cbd`::moving from state preparing -> state finished
>Thread-920::DEBUG::2012-12-09 12:24:37,811::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-920::DEBUG::2012-12-09 12:24:37,811::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-920::DEBUG::2012-12-09 12:24:37,811::task::978::TaskManager.Task::(_decref) Task=`fc546550-5964-4582-aee6-8eb0dbee6cbd`::ref 0 aborting False
>Thread-919::DEBUG::2012-12-09 12:24:37,882::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-919::WARNING::2012-12-09 12:24:37,884::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-919::DEBUG::2012-12-09 12:24:37,884::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-919::WARNING::2012-12-09 12:24:37,885::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-919::INFO::2012-12-09 12:24:37,886::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-919::DEBUG::2012-12-09 12:24:37,886::task::1172::TaskManager.Task::(prepare) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-919::DEBUG::2012-12-09 12:24:37,886::task::588::TaskManager.Task::(_updateState) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::moving from state preparing -> state finished
>Thread-919::DEBUG::2012-12-09 12:24:37,887::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-919::DEBUG::2012-12-09 12:24:37,887::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-919::DEBUG::2012-12-09 12:24:37,887::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-919::DEBUG::2012-12-09 12:24:37,888::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-919::DEBUG::2012-12-09 12:24:37,888::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-919::DEBUG::2012-12-09 12:24:37,888::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-919::DEBUG::2012-12-09 12:24:37,889::task::978::TaskManager.Task::(_decref) Task=`108f8d35-496a-46a5-adb2-508f4f126ab4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:24:39,541::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:39,586::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195171 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:41,593::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:41,639::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173227 s, 59.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:43,647::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:43,691::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197781 s, 51.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:43,784::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:43,785::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:24:43,899::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:43,900::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:43,900::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:43,900::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:24:45,698::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:45,741::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020284 s, 50.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:47,232::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:47,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000800064 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:47,748::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:47,798::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245147 s, 41.8 MB/s\n'; <rc> = 0
>Thread-926::DEBUG::2012-12-09 12:24:48,578::task::588::TaskManager.Task::(_updateState) Task=`20d24c8a-716a-4daa-b9aa-54f57646645a`::moving from state init -> state preparing
>Thread-926::INFO::2012-12-09 12:24:48,578::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-926::INFO::2012-12-09 12:24:48,578::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130159854889', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-926::DEBUG::2012-12-09 12:24:48,579::task::1172::TaskManager.Task::(prepare) Task=`20d24c8a-716a-4daa-b9aa-54f57646645a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130159854889', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-926::DEBUG::2012-12-09 12:24:48,579::task::588::TaskManager.Task::(_updateState) Task=`20d24c8a-716a-4daa-b9aa-54f57646645a`::moving from state preparing -> state finished
>Thread-926::DEBUG::2012-12-09 12:24:48,579::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-926::DEBUG::2012-12-09 12:24:48,579::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-926::DEBUG::2012-12-09 12:24:48,579::task::978::TaskManager.Task::(_decref) Task=`20d24c8a-716a-4daa-b9aa-54f57646645a`::ref 0 aborting False
>Thread-928::DEBUG::2012-12-09 12:24:48,653::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-928::DEBUG::2012-12-09 12:24:48,654::task::588::TaskManager.Task::(_updateState) Task=`c3aa2454-50de-4433-a94a-0aee39548ca0`::moving from state init -> state preparing
>Thread-928::INFO::2012-12-09 12:24:48,654::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-928::INFO::2012-12-09 12:24:48,654::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-928::DEBUG::2012-12-09 12:24:48,655::task::1172::TaskManager.Task::(prepare) Task=`c3aa2454-50de-4433-a94a-0aee39548ca0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-928::DEBUG::2012-12-09 12:24:48,655::task::588::TaskManager.Task::(_updateState) Task=`c3aa2454-50de-4433-a94a-0aee39548ca0`::moving from state preparing -> state finished
>Thread-928::DEBUG::2012-12-09 12:24:48,655::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-928::DEBUG::2012-12-09 12:24:48,655::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-928::DEBUG::2012-12-09 12:24:48,656::task::978::TaskManager.Task::(_decref) Task=`c3aa2454-50de-4433-a94a-0aee39548ca0`::ref 0 aborting False
>Thread-929::DEBUG::2012-12-09 12:24:48,669::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-929::DEBUG::2012-12-09 12:24:48,669::task::588::TaskManager.Task::(_updateState) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::moving from state init -> state preparing
>Thread-929::INFO::2012-12-09 12:24:48,669::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-929::DEBUG::2012-12-09 12:24:48,670::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5a1f7bde-82df-4861-913e-88a1582d428d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-929::DEBUG::2012-12-09 12:24:48,670::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-929::DEBUG::2012-12-09 12:24:48,670::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-929::DEBUG::2012-12-09 12:24:48,670::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5a1f7bde-82df-4861-913e-88a1582d428d`::Granted request
>Thread-929::DEBUG::2012-12-09 12:24:48,670::task::817::TaskManager.Task::(resourceAcquired) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-929::DEBUG::2012-12-09 12:24:48,671::task::978::TaskManager.Task::(_decref) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::ref 1 aborting False
>Thread-929::DEBUG::2012-12-09 12:24:48,671::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-929::DEBUG::2012-12-09 12:24:48,672::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-929::DEBUG::2012-12-09 12:24:48,765::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-929::WARNING::2012-12-09 12:24:48,766::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-929::DEBUG::2012-12-09 12:24:48,767::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-929::WARNING::2012-12-09 12:24:48,767::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-929::INFO::2012-12-09 12:24:48,768::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-929::DEBUG::2012-12-09 12:24:48,768::task::1172::TaskManager.Task::(prepare) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-929::DEBUG::2012-12-09 12:24:48,768::task::588::TaskManager.Task::(_updateState) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::moving from state preparing -> state finished
>Thread-929::DEBUG::2012-12-09 12:24:48,768::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-929::DEBUG::2012-12-09 12:24:48,768::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-929::DEBUG::2012-12-09 12:24:48,769::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-929::DEBUG::2012-12-09 12:24:48,769::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-929::DEBUG::2012-12-09 12:24:48,769::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-929::DEBUG::2012-12-09 12:24:48,769::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-929::DEBUG::2012-12-09 12:24:48,769::task::978::TaskManager.Task::(_decref) Task=`766b220e-26c3-4280-b4fa-6d4d798bfac5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:24:49,806::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:49,852::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217362 s, 47.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:51,858::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:51,896::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158971 s, 64.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:53,901::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:24:53,902::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:53,907::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:53,947::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157934 s, 64.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:24:54,010::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:24:54,011::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:24:54,012::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:24:54,012::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:24:55,954::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:56,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227486 s, 45.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:24:57,247::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:24:57,259::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000328603 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:24:58,008::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:24:58,054::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210833 s, 48.6 MB/s\n'; <rc> = 0
>Thread-934::DEBUG::2012-12-09 12:24:59,143::task::588::TaskManager.Task::(_updateState) Task=`112aa86c-7260-42a7-a9c7-cc0dbfbab59b`::moving from state init -> state preparing
>Thread-934::INFO::2012-12-09 12:24:59,144::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-934::INFO::2012-12-09 12:24:59,144::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.012286901474', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-934::DEBUG::2012-12-09 12:24:59,144::task::1172::TaskManager.Task::(prepare) Task=`112aa86c-7260-42a7-a9c7-cc0dbfbab59b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.012286901474', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-934::DEBUG::2012-12-09 12:24:59,144::task::588::TaskManager.Task::(_updateState) Task=`112aa86c-7260-42a7-a9c7-cc0dbfbab59b`::moving from state preparing -> state finished
>Thread-934::DEBUG::2012-12-09 12:24:59,144::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-934::DEBUG::2012-12-09 12:24:59,145::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-934::DEBUG::2012-12-09 12:24:59,145::task::978::TaskManager.Task::(_decref) Task=`112aa86c-7260-42a7-a9c7-cc0dbfbab59b`::ref 0 aborting False
>Thread-936::DEBUG::2012-12-09 12:24:59,302::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-936::DEBUG::2012-12-09 12:24:59,303::task::588::TaskManager.Task::(_updateState) Task=`8590ec29-bf42-4616-bdad-1e48b970b92a`::moving from state init -> state preparing
>Thread-936::INFO::2012-12-09 12:24:59,303::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-936::INFO::2012-12-09 12:24:59,303::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-936::DEBUG::2012-12-09 12:24:59,303::task::1172::TaskManager.Task::(prepare) Task=`8590ec29-bf42-4616-bdad-1e48b970b92a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-936::DEBUG::2012-12-09 12:24:59,304::task::588::TaskManager.Task::(_updateState) Task=`8590ec29-bf42-4616-bdad-1e48b970b92a`::moving from state preparing -> state finished
>Thread-936::DEBUG::2012-12-09 12:24:59,304::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-936::DEBUG::2012-12-09 12:24:59,304::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-936::DEBUG::2012-12-09 12:24:59,304::task::978::TaskManager.Task::(_decref) Task=`8590ec29-bf42-4616-bdad-1e48b970b92a`::ref 0 aborting False
>Thread-937::DEBUG::2012-12-09 12:24:59,317::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-937::DEBUG::2012-12-09 12:24:59,318::task::588::TaskManager.Task::(_updateState) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::moving from state init -> state preparing
>Thread-937::INFO::2012-12-09 12:24:59,318::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-937::DEBUG::2012-12-09 12:24:59,318::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2c57ca71-c993-4ec6-91bc-86b0daba135a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-937::DEBUG::2012-12-09 12:24:59,319::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-937::DEBUG::2012-12-09 12:24:59,319::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-937::DEBUG::2012-12-09 12:24:59,319::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2c57ca71-c993-4ec6-91bc-86b0daba135a`::Granted request
>Thread-937::DEBUG::2012-12-09 12:24:59,319::task::817::TaskManager.Task::(resourceAcquired) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-937::DEBUG::2012-12-09 12:24:59,320::task::978::TaskManager.Task::(_decref) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::ref 1 aborting False
>Thread-937::DEBUG::2012-12-09 12:24:59,320::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-937::DEBUG::2012-12-09 12:24:59,321::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-937::DEBUG::2012-12-09 12:24:59,417::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-937::WARNING::2012-12-09 12:24:59,419::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-937::DEBUG::2012-12-09 12:24:59,419::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-937::WARNING::2012-12-09 12:24:59,420::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-937::INFO::2012-12-09 12:24:59,420::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-937::DEBUG::2012-12-09 12:24:59,420::task::1172::TaskManager.Task::(prepare) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-937::DEBUG::2012-12-09 12:24:59,421::task::588::TaskManager.Task::(_updateState) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::moving from state preparing -> state finished
>Thread-937::DEBUG::2012-12-09 12:24:59,421::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-937::DEBUG::2012-12-09 12:24:59,421::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-937::DEBUG::2012-12-09 12:24:59,421::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-937::DEBUG::2012-12-09 12:24:59,421::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-937::DEBUG::2012-12-09 12:24:59,422::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-937::DEBUG::2012-12-09 12:24:59,422::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-937::DEBUG::2012-12-09 12:24:59,422::task::978::TaskManager.Task::(_decref) Task=`e5ee11c6-984c-49f7-9c78-32a385ec3659`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:00,060::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:00,102::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152544 s, 67.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:02,109::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:02,151::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018428 s, 55.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:25:04,013::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:04,014::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:04,114::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:04,115::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:04,116::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:04,116::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:25:04,158::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:04,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184749 s, 55.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:06,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:06,246::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016265 s, 63.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:07,262::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:07,276::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000675478 s, 6.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:08,253::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:08,299::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191882 s, 53.4 MB/s\n'; <rc> = 0
>Thread-942::DEBUG::2012-12-09 12:25:09,875::task::588::TaskManager.Task::(_updateState) Task=`36fc6cba-0dac-493c-b126-cd71060086a1`::moving from state init -> state preparing
>Thread-942::INFO::2012-12-09 12:25:09,876::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-942::INFO::2012-12-09 12:25:09,877::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151391029358', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.8', 'code': 358, 'valid': False}}
>Thread-942::DEBUG::2012-12-09 12:25:09,877::task::1172::TaskManager.Task::(prepare) Task=`36fc6cba-0dac-493c-b126-cd71060086a1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151391029358', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.8', 'code': 358, 'valid': False}}
>Thread-942::DEBUG::2012-12-09 12:25:09,877::task::588::TaskManager.Task::(_updateState) Task=`36fc6cba-0dac-493c-b126-cd71060086a1`::moving from state preparing -> state finished
>Thread-942::DEBUG::2012-12-09 12:25:09,878::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-942::DEBUG::2012-12-09 12:25:09,878::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-942::DEBUG::2012-12-09 12:25:09,878::task::978::TaskManager.Task::(_decref) Task=`36fc6cba-0dac-493c-b126-cd71060086a1`::ref 0 aborting False
>Thread-944::DEBUG::2012-12-09 12:25:10,286::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-944::DEBUG::2012-12-09 12:25:10,286::task::588::TaskManager.Task::(_updateState) Task=`fee7bcd1-dfa4-4f77-bf16-c2d2af53a228`::moving from state init -> state preparing
>Thread-944::INFO::2012-12-09 12:25:10,287::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-944::INFO::2012-12-09 12:25:10,287::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-944::DEBUG::2012-12-09 12:25:10,287::task::1172::TaskManager.Task::(prepare) Task=`fee7bcd1-dfa4-4f77-bf16-c2d2af53a228`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-944::DEBUG::2012-12-09 12:25:10,287::task::588::TaskManager.Task::(_updateState) Task=`fee7bcd1-dfa4-4f77-bf16-c2d2af53a228`::moving from state preparing -> state finished
>Thread-944::DEBUG::2012-12-09 12:25:10,288::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-944::DEBUG::2012-12-09 12:25:10,288::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-944::DEBUG::2012-12-09 12:25:10,288::task::978::TaskManager.Task::(_decref) Task=`fee7bcd1-dfa4-4f77-bf16-c2d2af53a228`::ref 0 aborting False
>Thread-945::DEBUG::2012-12-09 12:25:10,300::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-945::DEBUG::2012-12-09 12:25:10,300::task::588::TaskManager.Task::(_updateState) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::moving from state init -> state preparing
>Thread-945::INFO::2012-12-09 12:25:10,300::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-945::DEBUG::2012-12-09 12:25:10,301::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2918fcbe-48d3-45b0-a439-decb1089b023`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-945::DEBUG::2012-12-09 12:25:10,301::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-945::DEBUG::2012-12-09 12:25:10,301::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-945::DEBUG::2012-12-09 12:25:10,301::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2918fcbe-48d3-45b0-a439-decb1089b023`::Granted request
>Thread-945::DEBUG::2012-12-09 12:25:10,302::task::817::TaskManager.Task::(resourceAcquired) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-945::DEBUG::2012-12-09 12:25:10,302::task::978::TaskManager.Task::(_decref) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::ref 1 aborting False
>Thread-945::DEBUG::2012-12-09 12:25:10,302::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-945::DEBUG::2012-12-09 12:25:10,303::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:10,310::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:10,348::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158442 s, 64.6 MB/s\n'; <rc> = 0
>Thread-945::DEBUG::2012-12-09 12:25:10,398::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-945::WARNING::2012-12-09 12:25:10,400::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-945::DEBUG::2012-12-09 12:25:10,400::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-945::WARNING::2012-12-09 12:25:10,401::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-945::INFO::2012-12-09 12:25:10,402::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-945::DEBUG::2012-12-09 12:25:10,402::task::1172::TaskManager.Task::(prepare) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-945::DEBUG::2012-12-09 12:25:10,402::task::588::TaskManager.Task::(_updateState) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::moving from state preparing -> state finished
>Thread-945::DEBUG::2012-12-09 12:25:10,403::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-945::DEBUG::2012-12-09 12:25:10,403::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-945::DEBUG::2012-12-09 12:25:10,404::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-945::DEBUG::2012-12-09 12:25:10,404::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-945::DEBUG::2012-12-09 12:25:10,404::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-945::DEBUG::2012-12-09 12:25:10,405::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-945::DEBUG::2012-12-09 12:25:10,405::task::978::TaskManager.Task::(_decref) Task=`b4187eb4-0540-41f6-b99c-3878661a3502`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:12,355::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:12,400::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218332 s, 46.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:25:14,117::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:14,118::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:14,221::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:14,222::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:14,222::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:14,223::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:25:14,407::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:14,459::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239477 s, 42.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:16,466::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:16,509::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170762 s, 60.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:17,282::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:17,296::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000607635 s, 6.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:18,517::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:18,567::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251867 s, 40.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:20,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:20,616::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184071 s, 55.6 MB/s\n'; <rc> = 0
>Thread-950::DEBUG::2012-12-09 12:25:20,789::task::588::TaskManager.Task::(_updateState) Task=`a26e335f-fc85-45b2-ab9f-e4110e28f49d`::moving from state init -> state preparing
>Thread-950::INFO::2012-12-09 12:25:20,790::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-950::INFO::2012-12-09 12:25:20,790::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151062011719', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-950::DEBUG::2012-12-09 12:25:20,790::task::1172::TaskManager.Task::(prepare) Task=`a26e335f-fc85-45b2-ab9f-e4110e28f49d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151062011719', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-950::DEBUG::2012-12-09 12:25:20,791::task::588::TaskManager.Task::(_updateState) Task=`a26e335f-fc85-45b2-ab9f-e4110e28f49d`::moving from state preparing -> state finished
>Thread-950::DEBUG::2012-12-09 12:25:20,791::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-950::DEBUG::2012-12-09 12:25:20,791::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-950::DEBUG::2012-12-09 12:25:20,792::task::978::TaskManager.Task::(_decref) Task=`a26e335f-fc85-45b2-ab9f-e4110e28f49d`::ref 0 aborting False
>Thread-952::DEBUG::2012-12-09 12:25:21,010::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-952::DEBUG::2012-12-09 12:25:21,010::task::588::TaskManager.Task::(_updateState) Task=`4b5fede3-c41f-4f3e-8a03-643ee507b98a`::moving from state init -> state preparing
>Thread-952::INFO::2012-12-09 12:25:21,011::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-952::INFO::2012-12-09 12:25:21,011::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-952::DEBUG::2012-12-09 12:25:21,011::task::1172::TaskManager.Task::(prepare) Task=`4b5fede3-c41f-4f3e-8a03-643ee507b98a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-952::DEBUG::2012-12-09 12:25:21,012::task::588::TaskManager.Task::(_updateState) Task=`4b5fede3-c41f-4f3e-8a03-643ee507b98a`::moving from state preparing -> state finished
>Thread-952::DEBUG::2012-12-09 12:25:21,012::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-952::DEBUG::2012-12-09 12:25:21,012::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-952::DEBUG::2012-12-09 12:25:21,012::task::978::TaskManager.Task::(_decref) Task=`4b5fede3-c41f-4f3e-8a03-643ee507b98a`::ref 0 aborting False
>Thread-953::DEBUG::2012-12-09 12:25:21,026::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-953::DEBUG::2012-12-09 12:25:21,027::task::588::TaskManager.Task::(_updateState) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::moving from state init -> state preparing
>Thread-953::INFO::2012-12-09 12:25:21,027::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-953::DEBUG::2012-12-09 12:25:21,027::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f2abc193-204b-453e-a809-40d4a71efa89`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-953::DEBUG::2012-12-09 12:25:21,028::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-953::DEBUG::2012-12-09 12:25:21,028::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-953::DEBUG::2012-12-09 12:25:21,028::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f2abc193-204b-453e-a809-40d4a71efa89`::Granted request
>Thread-953::DEBUG::2012-12-09 12:25:21,029::task::817::TaskManager.Task::(resourceAcquired) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-953::DEBUG::2012-12-09 12:25:21,029::task::978::TaskManager.Task::(_decref) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::ref 1 aborting False
>Thread-953::DEBUG::2012-12-09 12:25:21,030::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-953::DEBUG::2012-12-09 12:25:21,030::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-953::DEBUG::2012-12-09 12:25:21,129::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-953::WARNING::2012-12-09 12:25:21,130::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-953::DEBUG::2012-12-09 12:25:21,130::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-953::WARNING::2012-12-09 12:25:21,131::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-953::INFO::2012-12-09 12:25:21,131::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-953::DEBUG::2012-12-09 12:25:21,131::task::1172::TaskManager.Task::(prepare) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-953::DEBUG::2012-12-09 12:25:21,132::task::588::TaskManager.Task::(_updateState) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::moving from state preparing -> state finished
>Thread-953::DEBUG::2012-12-09 12:25:21,132::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-953::DEBUG::2012-12-09 12:25:21,132::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-953::DEBUG::2012-12-09 12:25:21,132::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-953::DEBUG::2012-12-09 12:25:21,133::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-953::DEBUG::2012-12-09 12:25:21,133::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-953::DEBUG::2012-12-09 12:25:21,133::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-953::DEBUG::2012-12-09 12:25:21,133::task::978::TaskManager.Task::(_decref) Task=`1f288d99-4298-4ecb-a336-66588717a5e5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:22,623::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:22,661::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155697 s, 65.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:25:24,224::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:24,224::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:24,323::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:24,324::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:24,325::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:24,325::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:25:24,668::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:24,709::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191333 s, 53.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:26,715::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:26,755::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168415 s, 60.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:27,301::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:27,312::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000443897 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:28,762::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:28,808::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203821 s, 50.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:30,816::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:30,855::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163451 s, 62.6 MB/s\n'; <rc> = 0
>Thread-958::DEBUG::2012-12-09 12:25:31,274::task::588::TaskManager.Task::(_updateState) Task=`cf421566-b913-4abd-8272-7901153c88c1`::moving from state init -> state preparing
>Thread-958::INFO::2012-12-09 12:25:31,274::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-958::INFO::2012-12-09 12:25:31,274::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119078159332', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-958::DEBUG::2012-12-09 12:25:31,275::task::1172::TaskManager.Task::(prepare) Task=`cf421566-b913-4abd-8272-7901153c88c1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119078159332', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-958::DEBUG::2012-12-09 12:25:31,275::task::588::TaskManager.Task::(_updateState) Task=`cf421566-b913-4abd-8272-7901153c88c1`::moving from state preparing -> state finished
>Thread-958::DEBUG::2012-12-09 12:25:31,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-958::DEBUG::2012-12-09 12:25:31,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-958::DEBUG::2012-12-09 12:25:31,275::task::978::TaskManager.Task::(_decref) Task=`cf421566-b913-4abd-8272-7901153c88c1`::ref 0 aborting False
>Thread-960::DEBUG::2012-12-09 12:25:31,714::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-960::DEBUG::2012-12-09 12:25:31,715::task::588::TaskManager.Task::(_updateState) Task=`3954c73f-209c-498d-999b-0a5650e74090`::moving from state init -> state preparing
>Thread-960::INFO::2012-12-09 12:25:31,715::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-960::INFO::2012-12-09 12:25:31,715::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-960::DEBUG::2012-12-09 12:25:31,716::task::1172::TaskManager.Task::(prepare) Task=`3954c73f-209c-498d-999b-0a5650e74090`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-960::DEBUG::2012-12-09 12:25:31,716::task::588::TaskManager.Task::(_updateState) Task=`3954c73f-209c-498d-999b-0a5650e74090`::moving from state preparing -> state finished
>Thread-960::DEBUG::2012-12-09 12:25:31,716::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-960::DEBUG::2012-12-09 12:25:31,716::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-960::DEBUG::2012-12-09 12:25:31,716::task::978::TaskManager.Task::(_decref) Task=`3954c73f-209c-498d-999b-0a5650e74090`::ref 0 aborting False
>Thread-961::DEBUG::2012-12-09 12:25:31,729::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-961::DEBUG::2012-12-09 12:25:31,729::task::588::TaskManager.Task::(_updateState) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::moving from state init -> state preparing
>Thread-961::INFO::2012-12-09 12:25:31,730::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-961::DEBUG::2012-12-09 12:25:31,730::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`868bd7ae-3226-421c-987a-e45abd085aa8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-961::DEBUG::2012-12-09 12:25:31,730::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-961::DEBUG::2012-12-09 12:25:31,730::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-961::DEBUG::2012-12-09 12:25:31,731::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`868bd7ae-3226-421c-987a-e45abd085aa8`::Granted request
>Thread-961::DEBUG::2012-12-09 12:25:31,731::task::817::TaskManager.Task::(resourceAcquired) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-961::DEBUG::2012-12-09 12:25:31,731::task::978::TaskManager.Task::(_decref) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::ref 1 aborting False
>Thread-961::DEBUG::2012-12-09 12:25:31,732::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-961::DEBUG::2012-12-09 12:25:31,732::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-961::DEBUG::2012-12-09 12:25:31,832::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-961::WARNING::2012-12-09 12:25:31,833::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-961::DEBUG::2012-12-09 12:25:31,834::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-961::WARNING::2012-12-09 12:25:31,834::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-961::INFO::2012-12-09 12:25:31,834::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-961::DEBUG::2012-12-09 12:25:31,835::task::1172::TaskManager.Task::(prepare) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-961::DEBUG::2012-12-09 12:25:31,835::task::588::TaskManager.Task::(_updateState) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::moving from state preparing -> state finished
>Thread-961::DEBUG::2012-12-09 12:25:31,835::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-961::DEBUG::2012-12-09 12:25:31,835::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-961::DEBUG::2012-12-09 12:25:31,835::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-961::DEBUG::2012-12-09 12:25:31,836::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-961::DEBUG::2012-12-09 12:25:31,836::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-961::DEBUG::2012-12-09 12:25:31,836::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-961::DEBUG::2012-12-09 12:25:31,836::task::978::TaskManager.Task::(_decref) Task=`124d2f66-7969-4195-8b07-2dddc8d102d3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:32,862::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:32,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188172 s, 54.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:25:34,326::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:34,327::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:34,424::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:34,426::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:34,426::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:34,426::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:25:34,912::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:34,959::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022061 s, 46.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:36,966::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:37,009::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194461 s, 52.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:37,315::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:37,324::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000589213 s, 7.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:39,015::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:39,058::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176635 s, 58.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:41,064::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:41,115::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221057 s, 46.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:43,122::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:43,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178521 s, 57.4 MB/s\n'; <rc> = 0
>Thread-966::DEBUG::2012-12-09 12:25:43,670::task::588::TaskManager.Task::(_updateState) Task=`2fa52d82-1bca-4064-8c96-a7e3af97243a`::moving from state init -> state preparing
>Thread-966::INFO::2012-12-09 12:25:43,670::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-966::INFO::2012-12-09 12:25:43,671::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00938606262207', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-966::DEBUG::2012-12-09 12:25:43,671::task::1172::TaskManager.Task::(prepare) Task=`2fa52d82-1bca-4064-8c96-a7e3af97243a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00938606262207', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-966::DEBUG::2012-12-09 12:25:43,671::task::588::TaskManager.Task::(_updateState) Task=`2fa52d82-1bca-4064-8c96-a7e3af97243a`::moving from state preparing -> state finished
>Thread-966::DEBUG::2012-12-09 12:25:43,671::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-966::DEBUG::2012-12-09 12:25:43,672::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-966::DEBUG::2012-12-09 12:25:43,672::task::978::TaskManager.Task::(_decref) Task=`2fa52d82-1bca-4064-8c96-a7e3af97243a`::ref 0 aborting False
>Thread-968::DEBUG::2012-12-09 12:25:43,727::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-968::DEBUG::2012-12-09 12:25:43,727::task::588::TaskManager.Task::(_updateState) Task=`9cb1142d-d322-4881-a7ea-33868ae90b7e`::moving from state init -> state preparing
>Thread-968::INFO::2012-12-09 12:25:43,728::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-968::INFO::2012-12-09 12:25:43,728::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-968::DEBUG::2012-12-09 12:25:43,728::task::1172::TaskManager.Task::(prepare) Task=`9cb1142d-d322-4881-a7ea-33868ae90b7e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-968::DEBUG::2012-12-09 12:25:43,729::task::588::TaskManager.Task::(_updateState) Task=`9cb1142d-d322-4881-a7ea-33868ae90b7e`::moving from state preparing -> state finished
>Thread-968::DEBUG::2012-12-09 12:25:43,729::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-968::DEBUG::2012-12-09 12:25:43,729::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-968::DEBUG::2012-12-09 12:25:43,729::task::978::TaskManager.Task::(_decref) Task=`9cb1142d-d322-4881-a7ea-33868ae90b7e`::ref 0 aborting False
>Thread-969::DEBUG::2012-12-09 12:25:43,743::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-969::DEBUG::2012-12-09 12:25:43,744::task::588::TaskManager.Task::(_updateState) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::moving from state init -> state preparing
>Thread-969::INFO::2012-12-09 12:25:43,744::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-969::DEBUG::2012-12-09 12:25:43,745::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6588f90f-fe2f-4023-9f3d-ab5e2b23cf2b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-969::DEBUG::2012-12-09 12:25:43,745::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-969::DEBUG::2012-12-09 12:25:43,746::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-969::DEBUG::2012-12-09 12:25:43,746::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6588f90f-fe2f-4023-9f3d-ab5e2b23cf2b`::Granted request
>Thread-969::DEBUG::2012-12-09 12:25:43,746::task::817::TaskManager.Task::(resourceAcquired) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-969::DEBUG::2012-12-09 12:25:43,747::task::978::TaskManager.Task::(_decref) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::ref 1 aborting False
>Thread-969::DEBUG::2012-12-09 12:25:43,748::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-969::DEBUG::2012-12-09 12:25:43,748::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-969::DEBUG::2012-12-09 12:25:43,849::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-969::WARNING::2012-12-09 12:25:43,850::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-969::DEBUG::2012-12-09 12:25:43,851::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-969::WARNING::2012-12-09 12:25:43,851::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-969::INFO::2012-12-09 12:25:43,852::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-969::DEBUG::2012-12-09 12:25:43,852::task::1172::TaskManager.Task::(prepare) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-969::DEBUG::2012-12-09 12:25:43,852::task::588::TaskManager.Task::(_updateState) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::moving from state preparing -> state finished
>Thread-969::DEBUG::2012-12-09 12:25:43,852::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-969::DEBUG::2012-12-09 12:25:43,852::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-969::DEBUG::2012-12-09 12:25:43,853::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-969::DEBUG::2012-12-09 12:25:43,853::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-969::DEBUG::2012-12-09 12:25:43,853::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-969::DEBUG::2012-12-09 12:25:43,853::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-969::DEBUG::2012-12-09 12:25:43,854::task::978::TaskManager.Task::(_decref) Task=`50310712-1cf5-4eef-8202-068e57d10d82`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:25:44,428::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:44,428::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:44,528::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:44,529::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:44,529::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:44,530::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:25:45,173::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:45,223::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237393 s, 43.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:47,230::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:47,273::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189043 s, 54.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:47,328::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:47,341::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000825293 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:49,280::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:49,327::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189805 s, 54.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:51,334::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:53,433::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169254 s, 60.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:25:54,530::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:25:54,531::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:25:54,630::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:25:54,631::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:25:54,631::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:25:54,632::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-973::DEBUG::2012-12-09 12:25:55,208::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-973::DEBUG::2012-12-09 12:25:55,208::task::588::TaskManager.Task::(_updateState) Task=`13aef1b4-4383-4f21-815d-0587d306bc1f`::moving from state init -> state preparing
>Thread-973::INFO::2012-12-09 12:25:55,208::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-973::INFO::2012-12-09 12:25:55,209::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-973::DEBUG::2012-12-09 12:25:55,209::task::1172::TaskManager.Task::(prepare) Task=`13aef1b4-4383-4f21-815d-0587d306bc1f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-973::DEBUG::2012-12-09 12:25:55,209::task::588::TaskManager.Task::(_updateState) Task=`13aef1b4-4383-4f21-815d-0587d306bc1f`::moving from state preparing -> state finished
>Thread-973::DEBUG::2012-12-09 12:25:55,209::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-973::DEBUG::2012-12-09 12:25:55,209::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-973::DEBUG::2012-12-09 12:25:55,210::task::978::TaskManager.Task::(_decref) Task=`13aef1b4-4383-4f21-815d-0587d306bc1f`::ref 0 aborting False
>Thread-974::DEBUG::2012-12-09 12:25:55,222::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-974::DEBUG::2012-12-09 12:25:55,223::task::588::TaskManager.Task::(_updateState) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::moving from state init -> state preparing
>Thread-974::INFO::2012-12-09 12:25:55,223::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-974::DEBUG::2012-12-09 12:25:55,224::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2de39876-3505-48f9-8306-a2e3c32b6290`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-974::DEBUG::2012-12-09 12:25:55,225::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-974::DEBUG::2012-12-09 12:25:55,225::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-974::DEBUG::2012-12-09 12:25:55,226::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2de39876-3505-48f9-8306-a2e3c32b6290`::Granted request
>Thread-974::DEBUG::2012-12-09 12:25:55,226::task::817::TaskManager.Task::(resourceAcquired) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-974::DEBUG::2012-12-09 12:25:55,227::task::978::TaskManager.Task::(_decref) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::ref 1 aborting False
>Thread-974::DEBUG::2012-12-09 12:25:55,228::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-974::DEBUG::2012-12-09 12:25:55,229::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-974::DEBUG::2012-12-09 12:25:55,332::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-974::WARNING::2012-12-09 12:25:55,333::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-974::DEBUG::2012-12-09 12:25:55,333::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-974::WARNING::2012-12-09 12:25:55,334::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-974::INFO::2012-12-09 12:25:55,334::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-974::DEBUG::2012-12-09 12:25:55,334::task::1172::TaskManager.Task::(prepare) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-974::DEBUG::2012-12-09 12:25:55,335::task::588::TaskManager.Task::(_updateState) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::moving from state preparing -> state finished
>Thread-974::DEBUG::2012-12-09 12:25:55,335::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-974::DEBUG::2012-12-09 12:25:55,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-974::DEBUG::2012-12-09 12:25:55,336::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-974::DEBUG::2012-12-09 12:25:55,336::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-974::DEBUG::2012-12-09 12:25:55,336::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-974::DEBUG::2012-12-09 12:25:55,337::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-974::DEBUG::2012-12-09 12:25:55,337::task::978::TaskManager.Task::(_decref) Task=`8329b3ee-dfa5-49ce-b481-489bcc2f3c20`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:55,440::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:55,488::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228091 s, 44.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:25:57,345::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:25:57,356::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000453115 s, 9.0 MB/s\n'; <rc> = 0
>Thread-976::DEBUG::2012-12-09 12:25:57,448::task::588::TaskManager.Task::(_updateState) Task=`83366655-5a5a-4851-bbfe-a9ee891f2352`::moving from state init -> state preparing
>Thread-976::INFO::2012-12-09 12:25:57,449::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-976::INFO::2012-12-09 12:25:57,449::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118489265442', 'lastCheck': '0.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-976::DEBUG::2012-12-09 12:25:57,449::task::1172::TaskManager.Task::(prepare) Task=`83366655-5a5a-4851-bbfe-a9ee891f2352`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118489265442', 'lastCheck': '0.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-976::DEBUG::2012-12-09 12:25:57,450::task::588::TaskManager.Task::(_updateState) Task=`83366655-5a5a-4851-bbfe-a9ee891f2352`::moving from state preparing -> state finished
>Thread-976::DEBUG::2012-12-09 12:25:57,450::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-976::DEBUG::2012-12-09 12:25:57,450::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-976::DEBUG::2012-12-09 12:25:57,450::task::978::TaskManager.Task::(_decref) Task=`83366655-5a5a-4851-bbfe-a9ee891f2352`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:25:57,494::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:57,538::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179321 s, 57.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:25:59,545::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:25:59,592::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221705 s, 46.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:01,599::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:01,644::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172384 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:03,652::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:03,692::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016997 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:04,633::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:04,634::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:04,729::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:04,730::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:04,731::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:04,731::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:05,700::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:05,742::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185364 s, 55.2 MB/s\n'; <rc> = 0
>Thread-982::DEBUG::2012-12-09 12:26:06,124::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-982::DEBUG::2012-12-09 12:26:06,124::task::588::TaskManager.Task::(_updateState) Task=`1a0a27ec-330a-4fda-a37a-fbd42b11328f`::moving from state init -> state preparing
>Thread-982::INFO::2012-12-09 12:26:06,125::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-982::INFO::2012-12-09 12:26:06,125::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-982::DEBUG::2012-12-09 12:26:06,125::task::1172::TaskManager.Task::(prepare) Task=`1a0a27ec-330a-4fda-a37a-fbd42b11328f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-982::DEBUG::2012-12-09 12:26:06,125::task::588::TaskManager.Task::(_updateState) Task=`1a0a27ec-330a-4fda-a37a-fbd42b11328f`::moving from state preparing -> state finished
>Thread-982::DEBUG::2012-12-09 12:26:06,125::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-982::DEBUG::2012-12-09 12:26:06,126::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-982::DEBUG::2012-12-09 12:26:06,126::task::978::TaskManager.Task::(_decref) Task=`1a0a27ec-330a-4fda-a37a-fbd42b11328f`::ref 0 aborting False
>Thread-983::DEBUG::2012-12-09 12:26:06,143::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-983::DEBUG::2012-12-09 12:26:06,144::task::588::TaskManager.Task::(_updateState) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::moving from state init -> state preparing
>Thread-983::INFO::2012-12-09 12:26:06,144::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-983::DEBUG::2012-12-09 12:26:06,144::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1cd5bd29-c779-4c2e-82a8-74ca223a7a45`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-983::DEBUG::2012-12-09 12:26:06,145::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-983::DEBUG::2012-12-09 12:26:06,145::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-983::DEBUG::2012-12-09 12:26:06,145::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1cd5bd29-c779-4c2e-82a8-74ca223a7a45`::Granted request
>Thread-983::DEBUG::2012-12-09 12:26:06,145::task::817::TaskManager.Task::(resourceAcquired) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-983::DEBUG::2012-12-09 12:26:06,146::task::978::TaskManager.Task::(_decref) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::ref 1 aborting False
>Thread-983::DEBUG::2012-12-09 12:26:06,146::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-983::DEBUG::2012-12-09 12:26:06,147::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-983::DEBUG::2012-12-09 12:26:06,255::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-983::WARNING::2012-12-09 12:26:06,256::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-983::DEBUG::2012-12-09 12:26:06,256::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-983::WARNING::2012-12-09 12:26:06,257::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-983::INFO::2012-12-09 12:26:06,257::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-983::DEBUG::2012-12-09 12:26:06,257::task::1172::TaskManager.Task::(prepare) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-983::DEBUG::2012-12-09 12:26:06,257::task::588::TaskManager.Task::(_updateState) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::moving from state preparing -> state finished
>Thread-983::DEBUG::2012-12-09 12:26:06,258::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-983::DEBUG::2012-12-09 12:26:06,258::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-983::DEBUG::2012-12-09 12:26:06,258::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-983::DEBUG::2012-12-09 12:26:06,258::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-983::DEBUG::2012-12-09 12:26:06,259::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-983::DEBUG::2012-12-09 12:26:06,259::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-983::DEBUG::2012-12-09 12:26:06,259::task::978::TaskManager.Task::(_decref) Task=`6485e792-d1e1-4a36-911c-f0a0cbd271b4`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:26:07,359::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:07,370::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000438527 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:07,749::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:07,795::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0231456 s, 44.2 MB/s\n'; <rc> = 0
>Thread-984::DEBUG::2012-12-09 12:26:07,925::task::588::TaskManager.Task::(_updateState) Task=`30a64df9-272b-484a-88ba-c4eec1b9707f`::moving from state init -> state preparing
>Thread-984::INFO::2012-12-09 12:26:07,926::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-984::INFO::2012-12-09 12:26:07,926::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117321014404', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-984::DEBUG::2012-12-09 12:26:07,926::task::1172::TaskManager.Task::(prepare) Task=`30a64df9-272b-484a-88ba-c4eec1b9707f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117321014404', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-984::DEBUG::2012-12-09 12:26:07,926::task::588::TaskManager.Task::(_updateState) Task=`30a64df9-272b-484a-88ba-c4eec1b9707f`::moving from state preparing -> state finished
>Thread-984::DEBUG::2012-12-09 12:26:07,927::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-984::DEBUG::2012-12-09 12:26:07,927::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-984::DEBUG::2012-12-09 12:26:07,927::task::978::TaskManager.Task::(_decref) Task=`30a64df9-272b-484a-88ba-c4eec1b9707f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:26:09,802::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:09,843::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179938 s, 56.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:11,849::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:11,893::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204803 s, 50.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:13,900::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:13,943::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181032 s, 56.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:14,732::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:14,733::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:14,844::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:14,845::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:14,846::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:14,846::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:15,950::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:15,993::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191552 s, 53.5 MB/s\n'; <rc> = 0
>Thread-989::DEBUG::2012-12-09 12:26:17,066::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-989::DEBUG::2012-12-09 12:26:17,067::task::588::TaskManager.Task::(_updateState) Task=`fffaae19-2424-4823-b93c-f3e208af9490`::moving from state init -> state preparing
>Thread-989::INFO::2012-12-09 12:26:17,067::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-989::INFO::2012-12-09 12:26:17,068::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-989::DEBUG::2012-12-09 12:26:17,068::task::1172::TaskManager.Task::(prepare) Task=`fffaae19-2424-4823-b93c-f3e208af9490`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-989::DEBUG::2012-12-09 12:26:17,068::task::588::TaskManager.Task::(_updateState) Task=`fffaae19-2424-4823-b93c-f3e208af9490`::moving from state preparing -> state finished
>Thread-989::DEBUG::2012-12-09 12:26:17,068::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-989::DEBUG::2012-12-09 12:26:17,069::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-989::DEBUG::2012-12-09 12:26:17,069::task::978::TaskManager.Task::(_decref) Task=`fffaae19-2424-4823-b93c-f3e208af9490`::ref 0 aborting False
>Thread-991::DEBUG::2012-12-09 12:26:17,081::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-991::DEBUG::2012-12-09 12:26:17,082::task::588::TaskManager.Task::(_updateState) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::moving from state init -> state preparing
>Thread-991::INFO::2012-12-09 12:26:17,082::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-991::DEBUG::2012-12-09 12:26:17,083::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`44f58ddc-2691-40f8-afc0-4b269111de9a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-991::DEBUG::2012-12-09 12:26:17,083::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-991::DEBUG::2012-12-09 12:26:17,083::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-991::DEBUG::2012-12-09 12:26:17,083::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`44f58ddc-2691-40f8-afc0-4b269111de9a`::Granted request
>Thread-991::DEBUG::2012-12-09 12:26:17,084::task::817::TaskManager.Task::(resourceAcquired) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-991::DEBUG::2012-12-09 12:26:17,084::task::978::TaskManager.Task::(_decref) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::ref 1 aborting False
>Thread-991::DEBUG::2012-12-09 12:26:17,085::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-991::DEBUG::2012-12-09 12:26:17,085::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-991::DEBUG::2012-12-09 12:26:17,183::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-991::WARNING::2012-12-09 12:26:17,184::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-991::DEBUG::2012-12-09 12:26:17,184::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-991::WARNING::2012-12-09 12:26:17,185::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-991::INFO::2012-12-09 12:26:17,186::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-991::DEBUG::2012-12-09 12:26:17,186::task::1172::TaskManager.Task::(prepare) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-991::DEBUG::2012-12-09 12:26:17,186::task::588::TaskManager.Task::(_updateState) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::moving from state preparing -> state finished
>Thread-991::DEBUG::2012-12-09 12:26:17,186::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-991::DEBUG::2012-12-09 12:26:17,187::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-991::DEBUG::2012-12-09 12:26:17,187::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-991::DEBUG::2012-12-09 12:26:17,187::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-991::DEBUG::2012-12-09 12:26:17,187::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-991::DEBUG::2012-12-09 12:26:17,188::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-991::DEBUG::2012-12-09 12:26:17,188::task::978::TaskManager.Task::(_decref) Task=`f354da89-9110-49f1-8b2c-84068b40bb2b`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:26:17,373::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:17,388::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000814008 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:17,999::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:18,041::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178071 s, 57.5 MB/s\n'; <rc> = 0
>Thread-992::DEBUG::2012-12-09 12:26:19,109::task::588::TaskManager.Task::(_updateState) Task=`6b53a5c4-d0e3-464d-b74d-e89ea26cd6df`::moving from state init -> state preparing
>Thread-992::INFO::2012-12-09 12:26:19,109::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-992::INFO::2012-12-09 12:26:19,109::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160460472107', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-992::DEBUG::2012-12-09 12:26:19,110::task::1172::TaskManager.Task::(prepare) Task=`6b53a5c4-d0e3-464d-b74d-e89ea26cd6df`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160460472107', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-992::DEBUG::2012-12-09 12:26:19,110::task::588::TaskManager.Task::(_updateState) Task=`6b53a5c4-d0e3-464d-b74d-e89ea26cd6df`::moving from state preparing -> state finished
>Thread-992::DEBUG::2012-12-09 12:26:19,110::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-992::DEBUG::2012-12-09 12:26:19,110::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-992::DEBUG::2012-12-09 12:26:19,110::task::978::TaskManager.Task::(_decref) Task=`6b53a5c4-d0e3-464d-b74d-e89ea26cd6df`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:26:20,048::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:20,093::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168074 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:22,101::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:22,154::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265173 s, 38.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:24,161::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:24,210::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224039 s, 45.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:24,847::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:24,848::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:24,943::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:24,944::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:24,945::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:24,945::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:26,217::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:26,262::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193163 s, 53.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:26:27,393::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:27,404::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000990736 s, 4.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:28,269::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:28,312::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192403 s, 53.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:30,319::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:30,371::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261806 s, 39.1 MB/s\n'; <rc> = 0
>Thread-998::DEBUG::2012-12-09 12:26:31,158::task::588::TaskManager.Task::(_updateState) Task=`8d109733-7ee5-42a9-94e8-9efece411695`::moving from state init -> state preparing
>Thread-998::INFO::2012-12-09 12:26:31,159::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-998::INFO::2012-12-09 12:26:31,159::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126259326935', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-998::DEBUG::2012-12-09 12:26:31,159::task::1172::TaskManager.Task::(prepare) Task=`8d109733-7ee5-42a9-94e8-9efece411695`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126259326935', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-998::DEBUG::2012-12-09 12:26:31,159::task::588::TaskManager.Task::(_updateState) Task=`8d109733-7ee5-42a9-94e8-9efece411695`::moving from state preparing -> state finished
>Thread-998::DEBUG::2012-12-09 12:26:31,160::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-998::DEBUG::2012-12-09 12:26:31,160::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-998::DEBUG::2012-12-09 12:26:31,160::task::978::TaskManager.Task::(_decref) Task=`8d109733-7ee5-42a9-94e8-9efece411695`::ref 0 aborting False
>Thread-1000::DEBUG::2012-12-09 12:26:31,604::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1000::DEBUG::2012-12-09 12:26:31,604::task::588::TaskManager.Task::(_updateState) Task=`56b0cf75-b66c-4815-a474-b7cae5c6eba6`::moving from state init -> state preparing
>Thread-1000::INFO::2012-12-09 12:26:31,604::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1000::INFO::2012-12-09 12:26:31,605::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1000::DEBUG::2012-12-09 12:26:31,605::task::1172::TaskManager.Task::(prepare) Task=`56b0cf75-b66c-4815-a474-b7cae5c6eba6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1000::DEBUG::2012-12-09 12:26:31,605::task::588::TaskManager.Task::(_updateState) Task=`56b0cf75-b66c-4815-a474-b7cae5c6eba6`::moving from state preparing -> state finished
>Thread-1000::DEBUG::2012-12-09 12:26:31,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1000::DEBUG::2012-12-09 12:26:31,605::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1000::DEBUG::2012-12-09 12:26:31,606::task::978::TaskManager.Task::(_decref) Task=`56b0cf75-b66c-4815-a474-b7cae5c6eba6`::ref 0 aborting False
>Thread-1001::DEBUG::2012-12-09 12:26:31,624::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1001::DEBUG::2012-12-09 12:26:31,625::task::588::TaskManager.Task::(_updateState) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::moving from state init -> state preparing
>Thread-1001::INFO::2012-12-09 12:26:31,626::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1001::DEBUG::2012-12-09 12:26:31,626::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`59b9adfc-e225-4712-9fcc-0c6c9823741a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1001::DEBUG::2012-12-09 12:26:31,627::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1001::DEBUG::2012-12-09 12:26:31,627::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1001::DEBUG::2012-12-09 12:26:31,628::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`59b9adfc-e225-4712-9fcc-0c6c9823741a`::Granted request
>Thread-1001::DEBUG::2012-12-09 12:26:31,632::task::817::TaskManager.Task::(resourceAcquired) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1001::DEBUG::2012-12-09 12:26:31,632::task::978::TaskManager.Task::(_decref) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::ref 1 aborting False
>Thread-1001::DEBUG::2012-12-09 12:26:31,633::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1001::DEBUG::2012-12-09 12:26:31,633::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1001::DEBUG::2012-12-09 12:26:31,737::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1001::WARNING::2012-12-09 12:26:31,739::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1001::DEBUG::2012-12-09 12:26:31,739::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1001::WARNING::2012-12-09 12:26:31,740::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1001::INFO::2012-12-09 12:26:31,741::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1001::DEBUG::2012-12-09 12:26:31,741::task::1172::TaskManager.Task::(prepare) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1001::DEBUG::2012-12-09 12:26:31,741::task::588::TaskManager.Task::(_updateState) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::moving from state preparing -> state finished
>Thread-1001::DEBUG::2012-12-09 12:26:31,742::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1001::DEBUG::2012-12-09 12:26:31,742::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1001::DEBUG::2012-12-09 12:26:31,742::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1001::DEBUG::2012-12-09 12:26:31,742::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1001::DEBUG::2012-12-09 12:26:31,742::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1001::DEBUG::2012-12-09 12:26:31,743::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1001::DEBUG::2012-12-09 12:26:31,743::task::978::TaskManager.Task::(_decref) Task=`47e5b40f-d3f4-4b05-a272-0457497ca652`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:26:32,378::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:32,420::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179087 s, 57.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:34,427::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:34,467::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165491 s, 61.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:34,946::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:34,947::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:35,062::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:35,065::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:35,065::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:35,066::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:36,474::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:36,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191333 s, 53.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:26:37,410::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:37,424::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00078801 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:38,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:38,574::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272425 s, 37.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:40,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:40,620::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172848 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:42,627::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:42,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196852 s, 52.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:44,677::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:44,717::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01744 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:45,068::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:45,069::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:45,168::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:45,169::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:45,169::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:45,170::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:46,724::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:46,776::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265712 s, 38.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:26:47,429::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:47,443::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000871599 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:48,782::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:48,827::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200695 s, 51.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:50,833::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:50,878::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188933 s, 54.2 MB/s\n'; <rc> = 0
>Thread-1006::DEBUG::2012-12-09 12:26:51,222::task::588::TaskManager.Task::(_updateState) Task=`6a2346b6-3fdc-441b-a07c-931155665180`::moving from state init -> state preparing
>Thread-1006::INFO::2012-12-09 12:26:51,223::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1006::INFO::2012-12-09 12:26:51,223::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149128437042', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-1006::DEBUG::2012-12-09 12:26:51,224::task::1172::TaskManager.Task::(prepare) Task=`6a2346b6-3fdc-441b-a07c-931155665180`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149128437042', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-1006::DEBUG::2012-12-09 12:26:51,224::task::588::TaskManager.Task::(_updateState) Task=`6a2346b6-3fdc-441b-a07c-931155665180`::moving from state preparing -> state finished
>Thread-1006::DEBUG::2012-12-09 12:26:51,225::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1006::DEBUG::2012-12-09 12:26:51,225::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1006::DEBUG::2012-12-09 12:26:51,225::task::978::TaskManager.Task::(_decref) Task=`6a2346b6-3fdc-441b-a07c-931155665180`::ref 0 aborting False
>Thread-1008::DEBUG::2012-12-09 12:26:51,629::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1008::DEBUG::2012-12-09 12:26:51,630::task::588::TaskManager.Task::(_updateState) Task=`5aa391a8-52da-45e6-9231-2f7d8b7389b9`::moving from state init -> state preparing
>Thread-1008::INFO::2012-12-09 12:26:51,630::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1008::INFO::2012-12-09 12:26:51,630::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1008::DEBUG::2012-12-09 12:26:51,631::task::1172::TaskManager.Task::(prepare) Task=`5aa391a8-52da-45e6-9231-2f7d8b7389b9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1008::DEBUG::2012-12-09 12:26:51,631::task::588::TaskManager.Task::(_updateState) Task=`5aa391a8-52da-45e6-9231-2f7d8b7389b9`::moving from state preparing -> state finished
>Thread-1008::DEBUG::2012-12-09 12:26:51,631::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1008::DEBUG::2012-12-09 12:26:51,631::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1008::DEBUG::2012-12-09 12:26:51,631::task::978::TaskManager.Task::(_decref) Task=`5aa391a8-52da-45e6-9231-2f7d8b7389b9`::ref 0 aborting False
>Thread-1009::DEBUG::2012-12-09 12:26:51,646::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1009::DEBUG::2012-12-09 12:26:51,646::task::588::TaskManager.Task::(_updateState) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::moving from state init -> state preparing
>Thread-1009::INFO::2012-12-09 12:26:51,647::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1009::DEBUG::2012-12-09 12:26:51,647::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d28cb35d-074e-4486-97e1-5d71e4811ae8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1009::DEBUG::2012-12-09 12:26:51,648::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1009::DEBUG::2012-12-09 12:26:51,648::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1009::DEBUG::2012-12-09 12:26:51,648::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d28cb35d-074e-4486-97e1-5d71e4811ae8`::Granted request
>Thread-1009::DEBUG::2012-12-09 12:26:51,649::task::817::TaskManager.Task::(resourceAcquired) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1009::DEBUG::2012-12-09 12:26:51,649::task::978::TaskManager.Task::(_decref) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::ref 1 aborting False
>Thread-1009::DEBUG::2012-12-09 12:26:51,650::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1009::DEBUG::2012-12-09 12:26:51,651::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1009::DEBUG::2012-12-09 12:26:51,751::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1009::WARNING::2012-12-09 12:26:51,753::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1009::DEBUG::2012-12-09 12:26:51,753::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1009::WARNING::2012-12-09 12:26:51,754::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1009::INFO::2012-12-09 12:26:51,755::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1009::DEBUG::2012-12-09 12:26:51,756::task::1172::TaskManager.Task::(prepare) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1009::DEBUG::2012-12-09 12:26:51,756::task::588::TaskManager.Task::(_updateState) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::moving from state preparing -> state finished
>Thread-1009::DEBUG::2012-12-09 12:26:51,757::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1009::DEBUG::2012-12-09 12:26:51,757::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1009::DEBUG::2012-12-09 12:26:51,758::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1009::DEBUG::2012-12-09 12:26:51,758::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1009::DEBUG::2012-12-09 12:26:51,759::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1009::DEBUG::2012-12-09 12:26:51,759::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1009::DEBUG::2012-12-09 12:26:51,760::task::978::TaskManager.Task::(_decref) Task=`9b365f4f-3055-4115-b3dc-25da52314218`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:26:52,884::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:52,923::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016438 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:54,930::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:54,973::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179484 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:26:55,170::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:26:55,171::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:26:55,274::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:26:55,276::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:26:55,276::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:26:55,277::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:26:56,980::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:57,017::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163035 s, 62.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:26:57,449::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:26:57,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639067 s, 6.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:26:59,024::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:26:59,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158475 s, 64.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:01,071::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:01,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222808 s, 46.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:03,124::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:03,175::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258107 s, 39.7 MB/s\n'; <rc> = 0
>Thread-1014::DEBUG::2012-12-09 12:27:04,816::task::588::TaskManager.Task::(_updateState) Task=`4a37b0e3-6893-4471-adc1-676a66506a71`::moving from state init -> state preparing
>Thread-1014::INFO::2012-12-09 12:27:04,817::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1014::INFO::2012-12-09 12:27:04,817::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129930973053', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-1014::DEBUG::2012-12-09 12:27:04,818::task::1172::TaskManager.Task::(prepare) Task=`4a37b0e3-6893-4471-adc1-676a66506a71`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129930973053', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-1014::DEBUG::2012-12-09 12:27:04,818::task::588::TaskManager.Task::(_updateState) Task=`4a37b0e3-6893-4471-adc1-676a66506a71`::moving from state preparing -> state finished
>Thread-1014::DEBUG::2012-12-09 12:27:04,818::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1014::DEBUG::2012-12-09 12:27:04,819::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1014::DEBUG::2012-12-09 12:27:04,819::task::978::TaskManager.Task::(_decref) Task=`4a37b0e3-6893-4471-adc1-676a66506a71`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:27:05,182::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:05,235::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272097 s, 37.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:27:05,278::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:05,279::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:27:05,382::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:05,383::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:05,383::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:05,383::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1016::DEBUG::2012-12-09 12:27:05,534::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1016::DEBUG::2012-12-09 12:27:05,534::task::588::TaskManager.Task::(_updateState) Task=`e4445808-d188-4a4d-ad7d-cc418351de17`::moving from state init -> state preparing
>Thread-1016::INFO::2012-12-09 12:27:05,535::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1016::INFO::2012-12-09 12:27:05,535::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1016::DEBUG::2012-12-09 12:27:05,535::task::1172::TaskManager.Task::(prepare) Task=`e4445808-d188-4a4d-ad7d-cc418351de17`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1016::DEBUG::2012-12-09 12:27:05,535::task::588::TaskManager.Task::(_updateState) Task=`e4445808-d188-4a4d-ad7d-cc418351de17`::moving from state preparing -> state finished
>Thread-1016::DEBUG::2012-12-09 12:27:05,535::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1016::DEBUG::2012-12-09 12:27:05,536::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1016::DEBUG::2012-12-09 12:27:05,536::task::978::TaskManager.Task::(_decref) Task=`e4445808-d188-4a4d-ad7d-cc418351de17`::ref 0 aborting False
>Thread-1017::DEBUG::2012-12-09 12:27:05,551::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1017::DEBUG::2012-12-09 12:27:05,552::task::588::TaskManager.Task::(_updateState) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::moving from state init -> state preparing
>Thread-1017::INFO::2012-12-09 12:27:05,552::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1017::DEBUG::2012-12-09 12:27:05,553::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`001e174c-ea34-426b-814a-5bcc024c9c4d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1017::DEBUG::2012-12-09 12:27:05,553::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1017::DEBUG::2012-12-09 12:27:05,554::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1017::DEBUG::2012-12-09 12:27:05,554::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`001e174c-ea34-426b-814a-5bcc024c9c4d`::Granted request
>Thread-1017::DEBUG::2012-12-09 12:27:05,555::task::817::TaskManager.Task::(resourceAcquired) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1017::DEBUG::2012-12-09 12:27:05,555::task::978::TaskManager.Task::(_decref) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::ref 1 aborting False
>Thread-1017::DEBUG::2012-12-09 12:27:05,557::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1017::DEBUG::2012-12-09 12:27:05,558::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1017::DEBUG::2012-12-09 12:27:05,673::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1017::WARNING::2012-12-09 12:27:05,675::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1017::DEBUG::2012-12-09 12:27:05,676::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1017::WARNING::2012-12-09 12:27:05,677::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1017::INFO::2012-12-09 12:27:05,677::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1017::DEBUG::2012-12-09 12:27:05,678::task::1172::TaskManager.Task::(prepare) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1017::DEBUG::2012-12-09 12:27:05,678::task::588::TaskManager.Task::(_updateState) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::moving from state preparing -> state finished
>Thread-1017::DEBUG::2012-12-09 12:27:05,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1017::DEBUG::2012-12-09 12:27:05,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1017::DEBUG::2012-12-09 12:27:05,680::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1017::DEBUG::2012-12-09 12:27:05,681::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1017::DEBUG::2012-12-09 12:27:05,681::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1017::DEBUG::2012-12-09 12:27:05,681::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1017::DEBUG::2012-12-09 12:27:05,681::task::978::TaskManager.Task::(_decref) Task=`c02a9ee7-2aa4-4778-8047-a7834f4ebba5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:27:07,242::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:07,294::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271466 s, 37.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:27:07,465::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:07,480::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000938823 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:09,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:09,346::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228783 s, 44.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:11,352::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:11,409::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310422 s, 33.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:13,415::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:13,459::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194318 s, 52.7 MB/s\n'; <rc> = 0
>Thread-1022::DEBUG::2012-12-09 12:27:15,075::task::588::TaskManager.Task::(_updateState) Task=`69d65cfd-9c4e-42fa-89ad-bdce6b59582c`::moving from state init -> state preparing
>Thread-1022::INFO::2012-12-09 12:27:15,075::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1022::INFO::2012-12-09 12:27:15,076::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163140296936', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1022::DEBUG::2012-12-09 12:27:15,076::task::1172::TaskManager.Task::(prepare) Task=`69d65cfd-9c4e-42fa-89ad-bdce6b59582c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163140296936', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1022::DEBUG::2012-12-09 12:27:15,077::task::588::TaskManager.Task::(_updateState) Task=`69d65cfd-9c4e-42fa-89ad-bdce6b59582c`::moving from state preparing -> state finished
>Thread-1022::DEBUG::2012-12-09 12:27:15,077::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1022::DEBUG::2012-12-09 12:27:15,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1022::DEBUG::2012-12-09 12:27:15,078::task::978::TaskManager.Task::(_decref) Task=`69d65cfd-9c4e-42fa-89ad-bdce6b59582c`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:27:15,384::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:15,386::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:15,465::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:27:15,495::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:15,496::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:15,496::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:15,497::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:27:15,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172603 s, 59.3 MB/s\n'; <rc> = 0
>Thread-1024::DEBUG::2012-12-09 12:27:16,013::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1024::DEBUG::2012-12-09 12:27:16,014::task::588::TaskManager.Task::(_updateState) Task=`56dc6065-7dd2-409b-a660-ff37d0f5ca92`::moving from state init -> state preparing
>Thread-1024::INFO::2012-12-09 12:27:16,015::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1024::INFO::2012-12-09 12:27:16,015::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1024::DEBUG::2012-12-09 12:27:16,016::task::1172::TaskManager.Task::(prepare) Task=`56dc6065-7dd2-409b-a660-ff37d0f5ca92`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1024::DEBUG::2012-12-09 12:27:16,016::task::588::TaskManager.Task::(_updateState) Task=`56dc6065-7dd2-409b-a660-ff37d0f5ca92`::moving from state preparing -> state finished
>Thread-1024::DEBUG::2012-12-09 12:27:16,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1024::DEBUG::2012-12-09 12:27:16,017::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1024::DEBUG::2012-12-09 12:27:16,017::task::978::TaskManager.Task::(_decref) Task=`56dc6065-7dd2-409b-a660-ff37d0f5ca92`::ref 0 aborting False
>Thread-1025::DEBUG::2012-12-09 12:27:16,037::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1025::DEBUG::2012-12-09 12:27:16,038::task::588::TaskManager.Task::(_updateState) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::moving from state init -> state preparing
>Thread-1025::INFO::2012-12-09 12:27:16,038::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1025::DEBUG::2012-12-09 12:27:16,039::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`01b0c35b-ac2a-48ad-851b-fc9f1323cef8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1025::DEBUG::2012-12-09 12:27:16,039::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1025::DEBUG::2012-12-09 12:27:16,040::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1025::DEBUG::2012-12-09 12:27:16,040::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`01b0c35b-ac2a-48ad-851b-fc9f1323cef8`::Granted request
>Thread-1025::DEBUG::2012-12-09 12:27:16,041::task::817::TaskManager.Task::(resourceAcquired) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1025::DEBUG::2012-12-09 12:27:16,041::task::978::TaskManager.Task::(_decref) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::ref 1 aborting False
>Thread-1025::DEBUG::2012-12-09 12:27:16,043::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1025::DEBUG::2012-12-09 12:27:16,044::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1025::DEBUG::2012-12-09 12:27:16,154::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1025::WARNING::2012-12-09 12:27:16,157::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1025::DEBUG::2012-12-09 12:27:16,157::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1025::WARNING::2012-12-09 12:27:16,158::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1025::INFO::2012-12-09 12:27:16,159::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1025::DEBUG::2012-12-09 12:27:16,160::task::1172::TaskManager.Task::(prepare) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1025::DEBUG::2012-12-09 12:27:16,160::task::588::TaskManager.Task::(_updateState) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::moving from state preparing -> state finished
>Thread-1025::DEBUG::2012-12-09 12:27:16,161::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1025::DEBUG::2012-12-09 12:27:16,161::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1025::DEBUG::2012-12-09 12:27:16,161::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1025::DEBUG::2012-12-09 12:27:16,162::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1025::DEBUG::2012-12-09 12:27:16,162::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1025::DEBUG::2012-12-09 12:27:16,162::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1025::DEBUG::2012-12-09 12:27:16,162::task::978::TaskManager.Task::(_decref) Task=`da1dfd98-dabb-4d73-b8f8-d034cf18926d`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:27:17,484::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:17,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000939812 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:17,513::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:17,565::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268422 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:19,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:19,625::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276749 s, 37.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:21,632::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:21,669::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158061 s, 64.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:23,676::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:23,720::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197942 s, 51.7 MB/s\n'; <rc> = 0
>Thread-1030::DEBUG::2012-12-09 12:27:25,364::task::588::TaskManager.Task::(_updateState) Task=`5fcaaddb-1241-4bb7-8e4a-dc4f70ec15ae`::moving from state init -> state preparing
>Thread-1030::INFO::2012-12-09 12:27:25,365::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1030::INFO::2012-12-09 12:27:25,365::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154139995575', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-1030::DEBUG::2012-12-09 12:27:25,365::task::1172::TaskManager.Task::(prepare) Task=`5fcaaddb-1241-4bb7-8e4a-dc4f70ec15ae`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154139995575', 'lastCheck': '7.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-1030::DEBUG::2012-12-09 12:27:25,365::task::588::TaskManager.Task::(_updateState) Task=`5fcaaddb-1241-4bb7-8e4a-dc4f70ec15ae`::moving from state preparing -> state finished
>Thread-1030::DEBUG::2012-12-09 12:27:25,365::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1030::DEBUG::2012-12-09 12:27:25,366::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1030::DEBUG::2012-12-09 12:27:25,366::task::978::TaskManager.Task::(_decref) Task=`5fcaaddb-1241-4bb7-8e4a-dc4f70ec15ae`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:27:25,498::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:25,499::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:27:25,611::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:25,612::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:25,612::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:25,613::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:27:25,726::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:25,776::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256997 s, 39.8 MB/s\n'; <rc> = 0
>Thread-1032::DEBUG::2012-12-09 12:27:26,271::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1032::DEBUG::2012-12-09 12:27:26,273::task::588::TaskManager.Task::(_updateState) Task=`6b87f8a5-6043-4093-ac8a-04033c68a529`::moving from state init -> state preparing
>Thread-1032::INFO::2012-12-09 12:27:26,273::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1032::INFO::2012-12-09 12:27:26,274::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1032::DEBUG::2012-12-09 12:27:26,274::task::1172::TaskManager.Task::(prepare) Task=`6b87f8a5-6043-4093-ac8a-04033c68a529`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1032::DEBUG::2012-12-09 12:27:26,275::task::588::TaskManager.Task::(_updateState) Task=`6b87f8a5-6043-4093-ac8a-04033c68a529`::moving from state preparing -> state finished
>Thread-1032::DEBUG::2012-12-09 12:27:26,275::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1032::DEBUG::2012-12-09 12:27:26,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1032::DEBUG::2012-12-09 12:27:26,276::task::978::TaskManager.Task::(_decref) Task=`6b87f8a5-6043-4093-ac8a-04033c68a529`::ref 0 aborting False
>Thread-1033::DEBUG::2012-12-09 12:27:26,301::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1033::DEBUG::2012-12-09 12:27:26,302::task::588::TaskManager.Task::(_updateState) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::moving from state init -> state preparing
>Thread-1033::INFO::2012-12-09 12:27:26,302::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1033::DEBUG::2012-12-09 12:27:26,303::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7778517a-8f77-4975-9cc1-f77dcecffccc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1033::DEBUG::2012-12-09 12:27:26,303::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1033::DEBUG::2012-12-09 12:27:26,304::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1033::DEBUG::2012-12-09 12:27:26,304::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7778517a-8f77-4975-9cc1-f77dcecffccc`::Granted request
>Thread-1033::DEBUG::2012-12-09 12:27:26,305::task::817::TaskManager.Task::(resourceAcquired) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1033::DEBUG::2012-12-09 12:27:26,305::task::978::TaskManager.Task::(_decref) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::ref 1 aborting False
>Thread-1033::DEBUG::2012-12-09 12:27:26,307::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1033::DEBUG::2012-12-09 12:27:26,308::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1033::DEBUG::2012-12-09 12:27:26,411::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1033::WARNING::2012-12-09 12:27:26,412::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1033::DEBUG::2012-12-09 12:27:26,412::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1033::WARNING::2012-12-09 12:27:26,413::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1033::INFO::2012-12-09 12:27:26,413::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1033::DEBUG::2012-12-09 12:27:26,413::task::1172::TaskManager.Task::(prepare) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1033::DEBUG::2012-12-09 12:27:26,413::task::588::TaskManager.Task::(_updateState) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::moving from state preparing -> state finished
>Thread-1033::DEBUG::2012-12-09 12:27:26,414::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1033::DEBUG::2012-12-09 12:27:26,414::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1033::DEBUG::2012-12-09 12:27:26,414::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1033::DEBUG::2012-12-09 12:27:26,414::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1033::DEBUG::2012-12-09 12:27:26,414::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1033::DEBUG::2012-12-09 12:27:26,415::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1033::DEBUG::2012-12-09 12:27:26,415::task::978::TaskManager.Task::(_decref) Task=`06787613-b8a3-4ddc-bd61-da897277668e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:27:27,503::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:27,514::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000541319 s, 7.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:27,783::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:27,837::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026825 s, 38.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:29,843::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:29,895::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269233 s, 38.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:31,902::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:31,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229816 s, 44.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:33,955::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:34,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213075 s, 48.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:27:35,614::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:35,615::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1038::DEBUG::2012-12-09 12:27:35,685::task::588::TaskManager.Task::(_updateState) Task=`1bc5e4d8-ae62-436b-b243-e17cf3a9d0a9`::moving from state init -> state preparing
>Thread-1038::INFO::2012-12-09 12:27:35,686::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1038::INFO::2012-12-09 12:27:35,686::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117778778076', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-1038::DEBUG::2012-12-09 12:27:35,686::task::1172::TaskManager.Task::(prepare) Task=`1bc5e4d8-ae62-436b-b243-e17cf3a9d0a9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117778778076', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-1038::DEBUG::2012-12-09 12:27:35,687::task::588::TaskManager.Task::(_updateState) Task=`1bc5e4d8-ae62-436b-b243-e17cf3a9d0a9`::moving from state preparing -> state finished
>Thread-1038::DEBUG::2012-12-09 12:27:35,687::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1038::DEBUG::2012-12-09 12:27:35,687::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1038::DEBUG::2012-12-09 12:27:35,688::task::978::TaskManager.Task::(_decref) Task=`1bc5e4d8-ae62-436b-b243-e17cf3a9d0a9`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:27:35,724::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:35,725::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:35,725::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:35,725::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:27:36,008::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:36,056::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241079 s, 42.5 MB/s\n'; <rc> = 0
>Thread-1040::DEBUG::2012-12-09 12:27:36,512::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1040::DEBUG::2012-12-09 12:27:36,513::task::588::TaskManager.Task::(_updateState) Task=`52bd6c6f-5282-4f52-8e5d-2b01b2f1face`::moving from state init -> state preparing
>Thread-1040::INFO::2012-12-09 12:27:36,514::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1040::INFO::2012-12-09 12:27:36,515::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1040::DEBUG::2012-12-09 12:27:36,515::task::1172::TaskManager.Task::(prepare) Task=`52bd6c6f-5282-4f52-8e5d-2b01b2f1face`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1040::DEBUG::2012-12-09 12:27:36,515::task::588::TaskManager.Task::(_updateState) Task=`52bd6c6f-5282-4f52-8e5d-2b01b2f1face`::moving from state preparing -> state finished
>Thread-1040::DEBUG::2012-12-09 12:27:36,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1040::DEBUG::2012-12-09 12:27:36,516::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1040::DEBUG::2012-12-09 12:27:36,517::task::978::TaskManager.Task::(_decref) Task=`52bd6c6f-5282-4f52-8e5d-2b01b2f1face`::ref 0 aborting False
>Thread-1041::DEBUG::2012-12-09 12:27:36,541::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1041::DEBUG::2012-12-09 12:27:36,542::task::588::TaskManager.Task::(_updateState) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::moving from state init -> state preparing
>Thread-1041::INFO::2012-12-09 12:27:36,542::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1041::DEBUG::2012-12-09 12:27:36,543::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`95c79479-4407-48ae-8cc2-a9c81334aa06`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1041::DEBUG::2012-12-09 12:27:36,543::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1041::DEBUG::2012-12-09 12:27:36,544::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1041::DEBUG::2012-12-09 12:27:36,544::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`95c79479-4407-48ae-8cc2-a9c81334aa06`::Granted request
>Thread-1041::DEBUG::2012-12-09 12:27:36,545::task::817::TaskManager.Task::(resourceAcquired) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1041::DEBUG::2012-12-09 12:27:36,545::task::978::TaskManager.Task::(_decref) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::ref 1 aborting False
>Thread-1041::DEBUG::2012-12-09 12:27:36,547::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1041::DEBUG::2012-12-09 12:27:36,548::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1041::DEBUG::2012-12-09 12:27:36,659::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1041::WARNING::2012-12-09 12:27:36,661::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1041::DEBUG::2012-12-09 12:27:36,661::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1041::WARNING::2012-12-09 12:27:36,662::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1041::INFO::2012-12-09 12:27:36,662::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1041::DEBUG::2012-12-09 12:27:36,662::task::1172::TaskManager.Task::(prepare) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1041::DEBUG::2012-12-09 12:27:36,663::task::588::TaskManager.Task::(_updateState) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::moving from state preparing -> state finished
>Thread-1041::DEBUG::2012-12-09 12:27:36,663::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1041::DEBUG::2012-12-09 12:27:36,663::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1041::DEBUG::2012-12-09 12:27:36,663::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1041::DEBUG::2012-12-09 12:27:36,664::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1041::DEBUG::2012-12-09 12:27:36,664::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1041::DEBUG::2012-12-09 12:27:36,664::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1041::DEBUG::2012-12-09 12:27:36,664::task::978::TaskManager.Task::(_decref) Task=`42d087dd-2475-40af-afda-2c1afe203cec`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:27:37,517::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:37,531::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000942966 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:38,063::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:38,110::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209386 s, 48.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:40,116::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:40,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258771 s, 39.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:42,173::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:42,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210226 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:44,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:44,270::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196236 s, 52.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:27:45,726::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:45,727::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:27:45,841::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:45,842::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:45,843::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:45,843::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1046::DEBUG::2012-12-09 12:27:45,997::task::588::TaskManager.Task::(_updateState) Task=`e5f77b0c-da90-4545-a03f-6f24fd2efd71`::moving from state init -> state preparing
>Thread-1046::INFO::2012-12-09 12:27:45,997::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1046::INFO::2012-12-09 12:27:45,998::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151360034943', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1046::DEBUG::2012-12-09 12:27:45,998::task::1172::TaskManager.Task::(prepare) Task=`e5f77b0c-da90-4545-a03f-6f24fd2efd71`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0151360034943', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1046::DEBUG::2012-12-09 12:27:45,999::task::588::TaskManager.Task::(_updateState) Task=`e5f77b0c-da90-4545-a03f-6f24fd2efd71`::moving from state preparing -> state finished
>Thread-1046::DEBUG::2012-12-09 12:27:45,999::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1046::DEBUG::2012-12-09 12:27:45,999::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1046::DEBUG::2012-12-09 12:27:46,000::task::978::TaskManager.Task::(_decref) Task=`e5f77b0c-da90-4545-a03f-6f24fd2efd71`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:27:46,276::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:46,319::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185464 s, 55.2 MB/s\n'; <rc> = 0
>Thread-1048::DEBUG::2012-12-09 12:27:46,753::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1048::DEBUG::2012-12-09 12:27:46,754::task::588::TaskManager.Task::(_updateState) Task=`d406bbd1-667e-4201-889a-00e1f7dc1d57`::moving from state init -> state preparing
>Thread-1048::INFO::2012-12-09 12:27:46,755::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1048::INFO::2012-12-09 12:27:46,755::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1048::DEBUG::2012-12-09 12:27:46,756::task::1172::TaskManager.Task::(prepare) Task=`d406bbd1-667e-4201-889a-00e1f7dc1d57`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1048::DEBUG::2012-12-09 12:27:46,756::task::588::TaskManager.Task::(_updateState) Task=`d406bbd1-667e-4201-889a-00e1f7dc1d57`::moving from state preparing -> state finished
>Thread-1048::DEBUG::2012-12-09 12:27:46,757::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1048::DEBUG::2012-12-09 12:27:46,757::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1048::DEBUG::2012-12-09 12:27:46,758::task::978::TaskManager.Task::(_decref) Task=`d406bbd1-667e-4201-889a-00e1f7dc1d57`::ref 0 aborting False
>Thread-1049::DEBUG::2012-12-09 12:27:46,781::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1049::DEBUG::2012-12-09 12:27:46,782::task::588::TaskManager.Task::(_updateState) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::moving from state init -> state preparing
>Thread-1049::INFO::2012-12-09 12:27:46,782::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1049::DEBUG::2012-12-09 12:27:46,783::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`950bf381-b531-46cf-9a99-84799e0bc0ec`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1049::DEBUG::2012-12-09 12:27:46,784::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1049::DEBUG::2012-12-09 12:27:46,784::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1049::DEBUG::2012-12-09 12:27:46,785::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`950bf381-b531-46cf-9a99-84799e0bc0ec`::Granted request
>Thread-1049::DEBUG::2012-12-09 12:27:46,785::task::817::TaskManager.Task::(resourceAcquired) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1049::DEBUG::2012-12-09 12:27:46,786::task::978::TaskManager.Task::(_decref) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::ref 1 aborting False
>Thread-1049::DEBUG::2012-12-09 12:27:46,787::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1049::DEBUG::2012-12-09 12:27:46,788::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1049::DEBUG::2012-12-09 12:27:46,902::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1049::WARNING::2012-12-09 12:27:46,904::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1049::DEBUG::2012-12-09 12:27:46,904::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1049::WARNING::2012-12-09 12:27:46,905::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1049::INFO::2012-12-09 12:27:46,906::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1049::DEBUG::2012-12-09 12:27:46,907::task::1172::TaskManager.Task::(prepare) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1049::DEBUG::2012-12-09 12:27:46,907::task::588::TaskManager.Task::(_updateState) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::moving from state preparing -> state finished
>Thread-1049::DEBUG::2012-12-09 12:27:46,908::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1049::DEBUG::2012-12-09 12:27:46,908::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1049::DEBUG::2012-12-09 12:27:46,909::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1049::DEBUG::2012-12-09 12:27:46,909::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1049::DEBUG::2012-12-09 12:27:46,909::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1049::DEBUG::2012-12-09 12:27:46,910::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1049::DEBUG::2012-12-09 12:27:46,910::task::978::TaskManager.Task::(_decref) Task=`75dcf688-68d3-476a-8dd2-db919328c753`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:27:47,535::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:47,547::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000986782 s, 4.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:48,326::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:48,369::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200665 s, 51.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:50,376::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:50,432::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0298238 s, 34.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:52,440::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:52,483::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017572 s, 58.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:54,489::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:54,537::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245451 s, 41.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:27:55,844::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:27:55,845::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:27:55,966::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:27:55,967::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:27:55,967::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:27:55,967::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1054::DEBUG::2012-12-09 12:27:56,258::task::588::TaskManager.Task::(_updateState) Task=`ef3c3389-3887-4bcf-81e8-f4bd207362e2`::moving from state init -> state preparing
>Thread-1054::INFO::2012-12-09 12:27:56,258::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1054::INFO::2012-12-09 12:27:56,258::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013885974884', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-1054::DEBUG::2012-12-09 12:27:56,259::task::1172::TaskManager.Task::(prepare) Task=`ef3c3389-3887-4bcf-81e8-f4bd207362e2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013885974884', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-1054::DEBUG::2012-12-09 12:27:56,259::task::588::TaskManager.Task::(_updateState) Task=`ef3c3389-3887-4bcf-81e8-f4bd207362e2`::moving from state preparing -> state finished
>Thread-1054::DEBUG::2012-12-09 12:27:56,259::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1054::DEBUG::2012-12-09 12:27:56,259::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1054::DEBUG::2012-12-09 12:27:56,259::task::978::TaskManager.Task::(_decref) Task=`ef3c3389-3887-4bcf-81e8-f4bd207362e2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:27:56,543::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:56,589::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202073 s, 50.7 MB/s\n'; <rc> = 0
>Thread-1056::DEBUG::2012-12-09 12:27:56,966::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1056::DEBUG::2012-12-09 12:27:56,967::task::588::TaskManager.Task::(_updateState) Task=`548aa5f7-0267-4a1b-a31c-cf64681bd0de`::moving from state init -> state preparing
>Thread-1056::INFO::2012-12-09 12:27:56,967::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1056::INFO::2012-12-09 12:27:56,968::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1056::DEBUG::2012-12-09 12:27:56,968::task::1172::TaskManager.Task::(prepare) Task=`548aa5f7-0267-4a1b-a31c-cf64681bd0de`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1056::DEBUG::2012-12-09 12:27:56,969::task::588::TaskManager.Task::(_updateState) Task=`548aa5f7-0267-4a1b-a31c-cf64681bd0de`::moving from state preparing -> state finished
>Thread-1056::DEBUG::2012-12-09 12:27:56,969::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1056::DEBUG::2012-12-09 12:27:56,970::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1056::DEBUG::2012-12-09 12:27:56,970::task::978::TaskManager.Task::(_decref) Task=`548aa5f7-0267-4a1b-a31c-cf64681bd0de`::ref 0 aborting False
>Thread-1057::DEBUG::2012-12-09 12:27:56,993::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1057::DEBUG::2012-12-09 12:27:56,993::task::588::TaskManager.Task::(_updateState) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::moving from state init -> state preparing
>Thread-1057::INFO::2012-12-09 12:27:56,994::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1057::DEBUG::2012-12-09 12:27:56,994::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ab2ce22-86e1-40ec-96bf-f26c79776174`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1057::DEBUG::2012-12-09 12:27:56,994::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1057::DEBUG::2012-12-09 12:27:56,995::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1057::DEBUG::2012-12-09 12:27:56,995::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ab2ce22-86e1-40ec-96bf-f26c79776174`::Granted request
>Thread-1057::DEBUG::2012-12-09 12:27:56,995::task::817::TaskManager.Task::(resourceAcquired) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1057::DEBUG::2012-12-09 12:27:56,995::task::978::TaskManager.Task::(_decref) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::ref 1 aborting False
>Thread-1057::DEBUG::2012-12-09 12:27:56,997::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1057::DEBUG::2012-12-09 12:27:56,998::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1057::DEBUG::2012-12-09 12:27:57,105::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1057::WARNING::2012-12-09 12:27:57,106::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1057::DEBUG::2012-12-09 12:27:57,107::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1057::WARNING::2012-12-09 12:27:57,107::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1057::INFO::2012-12-09 12:27:57,108::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1057::DEBUG::2012-12-09 12:27:57,108::task::1172::TaskManager.Task::(prepare) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1057::DEBUG::2012-12-09 12:27:57,108::task::588::TaskManager.Task::(_updateState) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::moving from state preparing -> state finished
>Thread-1057::DEBUG::2012-12-09 12:27:57,108::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1057::DEBUG::2012-12-09 12:27:57,108::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1057::DEBUG::2012-12-09 12:27:57,109::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1057::DEBUG::2012-12-09 12:27:57,109::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1057::DEBUG::2012-12-09 12:27:57,109::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1057::DEBUG::2012-12-09 12:27:57,109::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1057::DEBUG::2012-12-09 12:27:57,110::task::978::TaskManager.Task::(_decref) Task=`bcfbf047-a06b-482c-acf5-fec11b6b444c`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:27:57,553::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:27:57,567::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000836922 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:27:58,595::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:27:58,639::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020073 s, 51.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:00,646::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:00,684::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162292 s, 63.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:02,690::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:02,736::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233741 s, 43.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:04,743::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:04,788::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212392 s, 48.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:05,969::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:05,970::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:06,081::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:06,083::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:06,083::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:06,083::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1062::DEBUG::2012-12-09 12:28:06,594::task::588::TaskManager.Task::(_updateState) Task=`f5c6eef2-e52c-43b0-a00c-9d2c4db06c59`::moving from state init -> state preparing
>Thread-1062::INFO::2012-12-09 12:28:06,594::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1062::INFO::2012-12-09 12:28:06,595::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154769420624', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-1062::DEBUG::2012-12-09 12:28:06,595::task::1172::TaskManager.Task::(prepare) Task=`f5c6eef2-e52c-43b0-a00c-9d2c4db06c59`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154769420624', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-1062::DEBUG::2012-12-09 12:28:06,596::task::588::TaskManager.Task::(_updateState) Task=`f5c6eef2-e52c-43b0-a00c-9d2c4db06c59`::moving from state preparing -> state finished
>Thread-1062::DEBUG::2012-12-09 12:28:06,596::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1062::DEBUG::2012-12-09 12:28:06,596::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1062::DEBUG::2012-12-09 12:28:06,597::task::978::TaskManager.Task::(_decref) Task=`f5c6eef2-e52c-43b0-a00c-9d2c4db06c59`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:06,795::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:06,850::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197674 s, 51.8 MB/s\n'; <rc> = 0
>Thread-1064::DEBUG::2012-12-09 12:28:07,207::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1064::DEBUG::2012-12-09 12:28:07,208::task::588::TaskManager.Task::(_updateState) Task=`5af6c641-e572-4f17-be51-4a7eca1fdeb6`::moving from state init -> state preparing
>Thread-1064::INFO::2012-12-09 12:28:07,208::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1064::INFO::2012-12-09 12:28:07,209::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1064::DEBUG::2012-12-09 12:28:07,209::task::1172::TaskManager.Task::(prepare) Task=`5af6c641-e572-4f17-be51-4a7eca1fdeb6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1064::DEBUG::2012-12-09 12:28:07,210::task::588::TaskManager.Task::(_updateState) Task=`5af6c641-e572-4f17-be51-4a7eca1fdeb6`::moving from state preparing -> state finished
>Thread-1064::DEBUG::2012-12-09 12:28:07,210::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1064::DEBUG::2012-12-09 12:28:07,210::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1064::DEBUG::2012-12-09 12:28:07,211::task::978::TaskManager.Task::(_decref) Task=`5af6c641-e572-4f17-be51-4a7eca1fdeb6`::ref 0 aborting False
>Thread-1065::DEBUG::2012-12-09 12:28:07,234::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1065::DEBUG::2012-12-09 12:28:07,234::task::588::TaskManager.Task::(_updateState) Task=`274070a9-5622-4e47-b143-8c055819d55a`::moving from state init -> state preparing
>Thread-1065::INFO::2012-12-09 12:28:07,235::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1065::DEBUG::2012-12-09 12:28:07,236::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`33f2c1fe-9632-4080-9af2-9b842dfe944e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1065::DEBUG::2012-12-09 12:28:07,236::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1065::DEBUG::2012-12-09 12:28:07,237::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1065::DEBUG::2012-12-09 12:28:07,237::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`33f2c1fe-9632-4080-9af2-9b842dfe944e`::Granted request
>Thread-1065::DEBUG::2012-12-09 12:28:07,238::task::817::TaskManager.Task::(resourceAcquired) Task=`274070a9-5622-4e47-b143-8c055819d55a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1065::DEBUG::2012-12-09 12:28:07,238::task::978::TaskManager.Task::(_decref) Task=`274070a9-5622-4e47-b143-8c055819d55a`::ref 1 aborting False
>Thread-1065::DEBUG::2012-12-09 12:28:07,240::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1065::DEBUG::2012-12-09 12:28:07,241::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1065::DEBUG::2012-12-09 12:28:07,341::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1065::WARNING::2012-12-09 12:28:07,343::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1065::DEBUG::2012-12-09 12:28:07,343::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1065::WARNING::2012-12-09 12:28:07,344::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1065::INFO::2012-12-09 12:28:07,344::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1065::DEBUG::2012-12-09 12:28:07,345::task::1172::TaskManager.Task::(prepare) Task=`274070a9-5622-4e47-b143-8c055819d55a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1065::DEBUG::2012-12-09 12:28:07,345::task::588::TaskManager.Task::(_updateState) Task=`274070a9-5622-4e47-b143-8c055819d55a`::moving from state preparing -> state finished
>Thread-1065::DEBUG::2012-12-09 12:28:07,345::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1065::DEBUG::2012-12-09 12:28:07,346::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1065::DEBUG::2012-12-09 12:28:07,347::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1065::DEBUG::2012-12-09 12:28:07,347::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1065::DEBUG::2012-12-09 12:28:07,347::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1065::DEBUG::2012-12-09 12:28:07,348::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1065::DEBUG::2012-12-09 12:28:07,348::task::978::TaskManager.Task::(_decref) Task=`274070a9-5622-4e47-b143-8c055819d55a`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:28:07,573::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:07,588::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000901107 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:08,856::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:08,906::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255299 s, 40.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:10,913::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:10,964::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027408 s, 37.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:12,970::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:13,015::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217714 s, 47.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:15,021::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:15,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171963 s, 59.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:16,084::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:16,085::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:16,201::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:16,202::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:16,203::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:16,203::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1070::DEBUG::2012-12-09 12:28:16,903::task::588::TaskManager.Task::(_updateState) Task=`06e5cbdf-f2d8-4184-89d6-adbf26d578ee`::moving from state init -> state preparing
>Thread-1070::INFO::2012-12-09 12:28:16,904::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1070::INFO::2012-12-09 12:28:16,904::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168631076813', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-1070::DEBUG::2012-12-09 12:28:16,904::task::1172::TaskManager.Task::(prepare) Task=`06e5cbdf-f2d8-4184-89d6-adbf26d578ee`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168631076813', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-1070::DEBUG::2012-12-09 12:28:16,904::task::588::TaskManager.Task::(_updateState) Task=`06e5cbdf-f2d8-4184-89d6-adbf26d578ee`::moving from state preparing -> state finished
>Thread-1070::DEBUG::2012-12-09 12:28:16,905::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1070::DEBUG::2012-12-09 12:28:16,905::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1070::DEBUG::2012-12-09 12:28:16,905::task::978::TaskManager.Task::(_decref) Task=`06e5cbdf-f2d8-4184-89d6-adbf26d578ee`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:17,068::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:17,114::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209894 s, 48.8 MB/s\n'; <rc> = 0
>Thread-1072::DEBUG::2012-12-09 12:28:17,565::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1072::DEBUG::2012-12-09 12:28:17,566::task::588::TaskManager.Task::(_updateState) Task=`4ccbcd4e-3cfa-4258-bbea-47b17b5b7f28`::moving from state init -> state preparing
>Thread-1072::INFO::2012-12-09 12:28:17,566::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1072::INFO::2012-12-09 12:28:17,567::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1072::DEBUG::2012-12-09 12:28:17,568::task::1172::TaskManager.Task::(prepare) Task=`4ccbcd4e-3cfa-4258-bbea-47b17b5b7f28`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1072::DEBUG::2012-12-09 12:28:17,568::task::588::TaskManager.Task::(_updateState) Task=`4ccbcd4e-3cfa-4258-bbea-47b17b5b7f28`::moving from state preparing -> state finished
>Thread-1072::DEBUG::2012-12-09 12:28:17,568::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1072::DEBUG::2012-12-09 12:28:17,569::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1072::DEBUG::2012-12-09 12:28:17,569::task::978::TaskManager.Task::(_decref) Task=`4ccbcd4e-3cfa-4258-bbea-47b17b5b7f28`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:28:17,593::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-1073::DEBUG::2012-12-09 12:28:17,600::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1073::DEBUG::2012-12-09 12:28:17,604::task::588::TaskManager.Task::(_updateState) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::moving from state init -> state preparing
>Thread-1073::INFO::2012-12-09 12:28:17,606::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1073::DEBUG::2012-12-09 12:28:17,609::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2d5b94c1-6475-4fd2-9bb7-3aff65a55fe6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1073::DEBUG::2012-12-09 12:28:17,610::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1073::DEBUG::2012-12-09 12:28:17,612::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-50::DEBUG::2012-12-09 12:28:17,613::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00080012 s, 5.1 MB/s\n'; <rc> = 0
>Thread-1073::DEBUG::2012-12-09 12:28:17,613::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2d5b94c1-6475-4fd2-9bb7-3aff65a55fe6`::Granted request
>Thread-1073::DEBUG::2012-12-09 12:28:17,616::task::817::TaskManager.Task::(resourceAcquired) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1073::DEBUG::2012-12-09 12:28:17,617::task::978::TaskManager.Task::(_decref) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::ref 1 aborting False
>Thread-1073::DEBUG::2012-12-09 12:28:17,621::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1073::DEBUG::2012-12-09 12:28:17,622::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1073::DEBUG::2012-12-09 12:28:17,718::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1073::WARNING::2012-12-09 12:28:17,719::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1073::DEBUG::2012-12-09 12:28:17,719::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1073::WARNING::2012-12-09 12:28:17,720::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1073::INFO::2012-12-09 12:28:17,720::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1073::DEBUG::2012-12-09 12:28:17,721::task::1172::TaskManager.Task::(prepare) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1073::DEBUG::2012-12-09 12:28:17,721::task::588::TaskManager.Task::(_updateState) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::moving from state preparing -> state finished
>Thread-1073::DEBUG::2012-12-09 12:28:17,721::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1073::DEBUG::2012-12-09 12:28:17,721::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1073::DEBUG::2012-12-09 12:28:17,722::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1073::DEBUG::2012-12-09 12:28:17,722::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1073::DEBUG::2012-12-09 12:28:17,722::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1073::DEBUG::2012-12-09 12:28:17,722::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1073::DEBUG::2012-12-09 12:28:17,723::task::978::TaskManager.Task::(_decref) Task=`7934051f-9ad8-4ab5-b27b-4d46ececffd6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:19,121::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:19,167::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021986 s, 46.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:21,174::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:21,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219602 s, 46.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:23,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:23,275::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248117 s, 41.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:25,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:25,329::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226808 s, 45.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:26,204::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:26,205::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:26,322::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:26,323::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:26,323::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:26,323::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1078::DEBUG::2012-12-09 12:28:27,184::task::588::TaskManager.Task::(_updateState) Task=`6763651a-c78d-40c5-a1e5-865389f8a6e8`::moving from state init -> state preparing
>Thread-1078::INFO::2012-12-09 12:28:27,184::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1078::INFO::2012-12-09 12:28:27,185::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0209269523621', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-1078::DEBUG::2012-12-09 12:28:27,185::task::1172::TaskManager.Task::(prepare) Task=`6763651a-c78d-40c5-a1e5-865389f8a6e8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0209269523621', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-1078::DEBUG::2012-12-09 12:28:27,186::task::588::TaskManager.Task::(_updateState) Task=`6763651a-c78d-40c5-a1e5-865389f8a6e8`::moving from state preparing -> state finished
>Thread-1078::DEBUG::2012-12-09 12:28:27,186::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1078::DEBUG::2012-12-09 12:28:27,187::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1078::DEBUG::2012-12-09 12:28:27,187::task::978::TaskManager.Task::(_decref) Task=`6763651a-c78d-40c5-a1e5-865389f8a6e8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:27,336::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:27,379::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198636 s, 51.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:28:27,627::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:27,641::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000973961 s, 4.2 MB/s\n'; <rc> = 0
>Thread-1080::DEBUG::2012-12-09 12:28:27,845::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1080::DEBUG::2012-12-09 12:28:27,846::task::588::TaskManager.Task::(_updateState) Task=`05192392-1dce-4443-a337-0d88c175bf04`::moving from state init -> state preparing
>Thread-1080::INFO::2012-12-09 12:28:27,847::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1080::INFO::2012-12-09 12:28:27,847::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1080::DEBUG::2012-12-09 12:28:27,848::task::1172::TaskManager.Task::(prepare) Task=`05192392-1dce-4443-a337-0d88c175bf04`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1080::DEBUG::2012-12-09 12:28:27,848::task::588::TaskManager.Task::(_updateState) Task=`05192392-1dce-4443-a337-0d88c175bf04`::moving from state preparing -> state finished
>Thread-1080::DEBUG::2012-12-09 12:28:27,848::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1080::DEBUG::2012-12-09 12:28:27,849::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1080::DEBUG::2012-12-09 12:28:27,849::task::978::TaskManager.Task::(_decref) Task=`05192392-1dce-4443-a337-0d88c175bf04`::ref 0 aborting False
>Thread-1081::DEBUG::2012-12-09 12:28:27,873::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1081::DEBUG::2012-12-09 12:28:27,874::task::588::TaskManager.Task::(_updateState) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::moving from state init -> state preparing
>Thread-1081::INFO::2012-12-09 12:28:27,874::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1081::DEBUG::2012-12-09 12:28:27,875::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cb535eb5-4a6c-417e-abce-6cf3e2e04a75`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1081::DEBUG::2012-12-09 12:28:27,876::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1081::DEBUG::2012-12-09 12:28:27,876::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1081::DEBUG::2012-12-09 12:28:27,876::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cb535eb5-4a6c-417e-abce-6cf3e2e04a75`::Granted request
>Thread-1081::DEBUG::2012-12-09 12:28:27,877::task::817::TaskManager.Task::(resourceAcquired) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1081::DEBUG::2012-12-09 12:28:27,878::task::978::TaskManager.Task::(_decref) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::ref 1 aborting False
>Thread-1081::DEBUG::2012-12-09 12:28:27,879::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1081::DEBUG::2012-12-09 12:28:27,880::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1081::DEBUG::2012-12-09 12:28:27,983::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1081::WARNING::2012-12-09 12:28:27,985::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1081::DEBUG::2012-12-09 12:28:27,985::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1081::WARNING::2012-12-09 12:28:27,986::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1081::INFO::2012-12-09 12:28:27,986::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1081::DEBUG::2012-12-09 12:28:27,987::task::1172::TaskManager.Task::(prepare) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1081::DEBUG::2012-12-09 12:28:27,987::task::588::TaskManager.Task::(_updateState) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::moving from state preparing -> state finished
>Thread-1081::DEBUG::2012-12-09 12:28:27,987::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1081::DEBUG::2012-12-09 12:28:27,987::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1081::DEBUG::2012-12-09 12:28:27,988::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1081::DEBUG::2012-12-09 12:28:27,988::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1081::DEBUG::2012-12-09 12:28:27,988::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1081::DEBUG::2012-12-09 12:28:27,988::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1081::DEBUG::2012-12-09 12:28:27,988::task::978::TaskManager.Task::(_decref) Task=`8ce76fdd-556e-44be-bc81-8a6301525409`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:29,386::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:29,435::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248071 s, 41.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:31,442::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:31,485::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171519 s, 59.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:33,492::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:33,540::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023439 s, 43.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:35,547::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:35,596::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025156 s, 40.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:36,324::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:36,325::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:36,442::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:36,443::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:36,443::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:36,444::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1086::DEBUG::2012-12-09 12:28:37,415::task::588::TaskManager.Task::(_updateState) Task=`8d5cd32d-23c8-40b5-b7f8-3cb2b9ebc72f`::moving from state init -> state preparing
>Thread-1086::INFO::2012-12-09 12:28:37,416::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1086::INFO::2012-12-09 12:28:37,416::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152630805969', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-1086::DEBUG::2012-12-09 12:28:37,417::task::1172::TaskManager.Task::(prepare) Task=`8d5cd32d-23c8-40b5-b7f8-3cb2b9ebc72f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152630805969', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-1086::DEBUG::2012-12-09 12:28:37,417::task::588::TaskManager.Task::(_updateState) Task=`8d5cd32d-23c8-40b5-b7f8-3cb2b9ebc72f`::moving from state preparing -> state finished
>Thread-1086::DEBUG::2012-12-09 12:28:37,418::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1086::DEBUG::2012-12-09 12:28:37,418::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1086::DEBUG::2012-12-09 12:28:37,419::task::978::TaskManager.Task::(_decref) Task=`8d5cd32d-23c8-40b5-b7f8-3cb2b9ebc72f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:37,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:37,646::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:37,660::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000367428 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:37,669::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215334 s, 47.6 MB/s\n'; <rc> = 0
>Thread-1088::DEBUG::2012-12-09 12:28:38,050::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1088::DEBUG::2012-12-09 12:28:38,050::task::588::TaskManager.Task::(_updateState) Task=`1a949310-17fe-4a6e-87cc-d0dc8b90bd63`::moving from state init -> state preparing
>Thread-1088::INFO::2012-12-09 12:28:38,051::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1088::INFO::2012-12-09 12:28:38,051::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1088::DEBUG::2012-12-09 12:28:38,051::task::1172::TaskManager.Task::(prepare) Task=`1a949310-17fe-4a6e-87cc-d0dc8b90bd63`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1088::DEBUG::2012-12-09 12:28:38,051::task::588::TaskManager.Task::(_updateState) Task=`1a949310-17fe-4a6e-87cc-d0dc8b90bd63`::moving from state preparing -> state finished
>Thread-1088::DEBUG::2012-12-09 12:28:38,052::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1088::DEBUG::2012-12-09 12:28:38,052::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1088::DEBUG::2012-12-09 12:28:38,052::task::978::TaskManager.Task::(_decref) Task=`1a949310-17fe-4a6e-87cc-d0dc8b90bd63`::ref 0 aborting False
>Thread-1089::DEBUG::2012-12-09 12:28:38,065::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1089::DEBUG::2012-12-09 12:28:38,066::task::588::TaskManager.Task::(_updateState) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::moving from state init -> state preparing
>Thread-1089::INFO::2012-12-09 12:28:38,066::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1089::DEBUG::2012-12-09 12:28:38,067::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9d9b7dfe-fcf9-4906-a86f-e7d9a2609e71`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1089::DEBUG::2012-12-09 12:28:38,067::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1089::DEBUG::2012-12-09 12:28:38,067::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1089::DEBUG::2012-12-09 12:28:38,067::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9d9b7dfe-fcf9-4906-a86f-e7d9a2609e71`::Granted request
>Thread-1089::DEBUG::2012-12-09 12:28:38,068::task::817::TaskManager.Task::(resourceAcquired) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1089::DEBUG::2012-12-09 12:28:38,068::task::978::TaskManager.Task::(_decref) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::ref 1 aborting False
>Thread-1089::DEBUG::2012-12-09 12:28:38,069::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1089::DEBUG::2012-12-09 12:28:38,069::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1089::DEBUG::2012-12-09 12:28:38,163::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1089::WARNING::2012-12-09 12:28:38,164::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1089::DEBUG::2012-12-09 12:28:38,164::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1089::WARNING::2012-12-09 12:28:38,165::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1089::INFO::2012-12-09 12:28:38,165::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1089::DEBUG::2012-12-09 12:28:38,165::task::1172::TaskManager.Task::(prepare) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1089::DEBUG::2012-12-09 12:28:38,166::task::588::TaskManager.Task::(_updateState) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::moving from state preparing -> state finished
>Thread-1089::DEBUG::2012-12-09 12:28:38,166::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1089::DEBUG::2012-12-09 12:28:38,166::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1089::DEBUG::2012-12-09 12:28:38,166::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1089::DEBUG::2012-12-09 12:28:38,167::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1089::DEBUG::2012-12-09 12:28:38,167::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1089::DEBUG::2012-12-09 12:28:38,167::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1089::DEBUG::2012-12-09 12:28:38,167::task::978::TaskManager.Task::(_decref) Task=`274160b0-c04d-412c-8fe5-6d715ec57619`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:39,678::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:39,733::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245212 s, 41.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:41,740::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:41,786::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238058 s, 43.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:43,792::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:43,838::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204903 s, 50.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:45,844::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:45,890::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213109 s, 48.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:46,445::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:46,446::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:46,558::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:46,559::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:46,560::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:46,560::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:28:47,675::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:47,688::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000703575 s, 5.8 MB/s\n'; <rc> = 0
>Thread-1094::DEBUG::2012-12-09 12:28:47,857::task::588::TaskManager.Task::(_updateState) Task=`b36fb9c0-a17f-408d-8559-30866411995c`::moving from state init -> state preparing
>Thread-1094::INFO::2012-12-09 12:28:47,857::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1094::INFO::2012-12-09 12:28:47,857::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127401351929', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-1094::DEBUG::2012-12-09 12:28:47,858::task::1172::TaskManager.Task::(prepare) Task=`b36fb9c0-a17f-408d-8559-30866411995c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127401351929', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-1094::DEBUG::2012-12-09 12:28:47,858::task::588::TaskManager.Task::(_updateState) Task=`b36fb9c0-a17f-408d-8559-30866411995c`::moving from state preparing -> state finished
>Thread-1094::DEBUG::2012-12-09 12:28:47,858::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1094::DEBUG::2012-12-09 12:28:47,858::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1094::DEBUG::2012-12-09 12:28:47,858::task::978::TaskManager.Task::(_decref) Task=`b36fb9c0-a17f-408d-8559-30866411995c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:47,896::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:47,945::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238795 s, 42.9 MB/s\n'; <rc> = 0
>Thread-1096::DEBUG::2012-12-09 12:28:48,274::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1096::DEBUG::2012-12-09 12:28:48,275::task::588::TaskManager.Task::(_updateState) Task=`eb5ebefa-c0ab-4dbe-a2ca-b5fae101c6cd`::moving from state init -> state preparing
>Thread-1096::INFO::2012-12-09 12:28:48,276::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1096::INFO::2012-12-09 12:28:48,277::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1096::DEBUG::2012-12-09 12:28:48,277::task::1172::TaskManager.Task::(prepare) Task=`eb5ebefa-c0ab-4dbe-a2ca-b5fae101c6cd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1096::DEBUG::2012-12-09 12:28:48,277::task::588::TaskManager.Task::(_updateState) Task=`eb5ebefa-c0ab-4dbe-a2ca-b5fae101c6cd`::moving from state preparing -> state finished
>Thread-1096::DEBUG::2012-12-09 12:28:48,278::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1096::DEBUG::2012-12-09 12:28:48,278::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1096::DEBUG::2012-12-09 12:28:48,279::task::978::TaskManager.Task::(_decref) Task=`eb5ebefa-c0ab-4dbe-a2ca-b5fae101c6cd`::ref 0 aborting False
>Thread-1097::DEBUG::2012-12-09 12:28:48,303::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1097::DEBUG::2012-12-09 12:28:48,304::task::588::TaskManager.Task::(_updateState) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::moving from state init -> state preparing
>Thread-1097::INFO::2012-12-09 12:28:48,304::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1097::DEBUG::2012-12-09 12:28:48,305::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c462e27-2a77-4b27-9d53-cf12d2c803f1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1097::DEBUG::2012-12-09 12:28:48,305::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1097::DEBUG::2012-12-09 12:28:48,306::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1097::DEBUG::2012-12-09 12:28:48,306::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c462e27-2a77-4b27-9d53-cf12d2c803f1`::Granted request
>Thread-1097::DEBUG::2012-12-09 12:28:48,307::task::817::TaskManager.Task::(resourceAcquired) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1097::DEBUG::2012-12-09 12:28:48,307::task::978::TaskManager.Task::(_decref) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::ref 1 aborting False
>Thread-1097::DEBUG::2012-12-09 12:28:48,309::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1097::DEBUG::2012-12-09 12:28:48,310::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1097::DEBUG::2012-12-09 12:28:48,421::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1097::WARNING::2012-12-09 12:28:48,422::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1097::DEBUG::2012-12-09 12:28:48,423::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1097::WARNING::2012-12-09 12:28:48,423::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1097::INFO::2012-12-09 12:28:48,423::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1097::DEBUG::2012-12-09 12:28:48,423::task::1172::TaskManager.Task::(prepare) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1097::DEBUG::2012-12-09 12:28:48,424::task::588::TaskManager.Task::(_updateState) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::moving from state preparing -> state finished
>Thread-1097::DEBUG::2012-12-09 12:28:48,424::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1097::DEBUG::2012-12-09 12:28:48,424::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1097::DEBUG::2012-12-09 12:28:48,424::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1097::DEBUG::2012-12-09 12:28:48,425::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1097::DEBUG::2012-12-09 12:28:48,425::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1097::DEBUG::2012-12-09 12:28:48,425::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1097::DEBUG::2012-12-09 12:28:48,425::task::978::TaskManager.Task::(_decref) Task=`1c8338eb-c6a0-4f37-8be7-7f14f8baba8e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:49,951::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:50,000::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176194 s, 58.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:52,007::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:52,055::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0253385 s, 40.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:54,061::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:54,116::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305737 s, 33.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:28:56,123::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:56,176::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0284169 s, 36.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:28:56,561::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:28:56,562::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:28:56,682::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:28:56,683::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:28:56,684::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:28:56,684::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:28:57,691::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:28:57,702::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000504398 s, 8.1 MB/s\n'; <rc> = 0
>Thread-1102::DEBUG::2012-12-09 12:28:58,146::task::588::TaskManager.Task::(_updateState) Task=`fbd6eee2-09bb-4815-b250-351f57863460`::moving from state init -> state preparing
>Thread-1102::INFO::2012-12-09 12:28:58,146::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1102::INFO::2012-12-09 12:28:58,147::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116150379181', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-1102::DEBUG::2012-12-09 12:28:58,147::task::1172::TaskManager.Task::(prepare) Task=`fbd6eee2-09bb-4815-b250-351f57863460`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116150379181', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-1102::DEBUG::2012-12-09 12:28:58,148::task::588::TaskManager.Task::(_updateState) Task=`fbd6eee2-09bb-4815-b250-351f57863460`::moving from state preparing -> state finished
>Thread-1102::DEBUG::2012-12-09 12:28:58,148::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1102::DEBUG::2012-12-09 12:28:58,149::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1102::DEBUG::2012-12-09 12:28:58,149::task::978::TaskManager.Task::(_decref) Task=`fbd6eee2-09bb-4815-b250-351f57863460`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:28:58,183::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:28:58,230::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220057 s, 46.5 MB/s\n'; <rc> = 0
>Thread-1104::DEBUG::2012-12-09 12:28:58,537::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1104::DEBUG::2012-12-09 12:28:58,537::task::588::TaskManager.Task::(_updateState) Task=`5c364058-db93-44e6-ae7e-d54467e52c9f`::moving from state init -> state preparing
>Thread-1104::INFO::2012-12-09 12:28:58,537::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1104::INFO::2012-12-09 12:28:58,538::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1104::DEBUG::2012-12-09 12:28:58,538::task::1172::TaskManager.Task::(prepare) Task=`5c364058-db93-44e6-ae7e-d54467e52c9f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1104::DEBUG::2012-12-09 12:28:58,538::task::588::TaskManager.Task::(_updateState) Task=`5c364058-db93-44e6-ae7e-d54467e52c9f`::moving from state preparing -> state finished
>Thread-1104::DEBUG::2012-12-09 12:28:58,538::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1104::DEBUG::2012-12-09 12:28:58,538::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1104::DEBUG::2012-12-09 12:28:58,539::task::978::TaskManager.Task::(_decref) Task=`5c364058-db93-44e6-ae7e-d54467e52c9f`::ref 0 aborting False
>Thread-1105::DEBUG::2012-12-09 12:28:58,554::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1105::DEBUG::2012-12-09 12:28:58,555::task::588::TaskManager.Task::(_updateState) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::moving from state init -> state preparing
>Thread-1105::INFO::2012-12-09 12:28:58,555::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1105::DEBUG::2012-12-09 12:28:58,556::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1a359623-3c79-4cf7-85d6-b15c173ff8c2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1105::DEBUG::2012-12-09 12:28:58,556::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1105::DEBUG::2012-12-09 12:28:58,557::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1105::DEBUG::2012-12-09 12:28:58,557::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1a359623-3c79-4cf7-85d6-b15c173ff8c2`::Granted request
>Thread-1105::DEBUG::2012-12-09 12:28:58,558::task::817::TaskManager.Task::(resourceAcquired) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1105::DEBUG::2012-12-09 12:28:58,558::task::978::TaskManager.Task::(_decref) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::ref 1 aborting False
>Thread-1105::DEBUG::2012-12-09 12:28:58,560::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1105::DEBUG::2012-12-09 12:28:58,561::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1105::DEBUG::2012-12-09 12:28:58,666::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1105::WARNING::2012-12-09 12:28:58,667::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1105::DEBUG::2012-12-09 12:28:58,667::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1105::WARNING::2012-12-09 12:28:58,668::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1105::INFO::2012-12-09 12:28:58,668::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1105::DEBUG::2012-12-09 12:28:58,669::task::1172::TaskManager.Task::(prepare) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1105::DEBUG::2012-12-09 12:28:58,669::task::588::TaskManager.Task::(_updateState) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::moving from state preparing -> state finished
>Thread-1105::DEBUG::2012-12-09 12:28:58,669::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1105::DEBUG::2012-12-09 12:28:58,669::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1105::DEBUG::2012-12-09 12:28:58,670::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1105::DEBUG::2012-12-09 12:28:58,670::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1105::DEBUG::2012-12-09 12:28:58,670::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1105::DEBUG::2012-12-09 12:28:58,670::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1105::DEBUG::2012-12-09 12:28:58,671::task::978::TaskManager.Task::(_decref) Task=`26c873c0-aaba-453a-91ab-c034803d3980`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:00,237::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:00,282::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199385 s, 51.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:02,290::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:02,336::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168924 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:04,342::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:04,383::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187544 s, 54.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:06,389::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:06,438::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254238 s, 40.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:06,685::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:06,686::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:29:06,800::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:06,802::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:06,802::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:06,802::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:29:07,704::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:29:07,705::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:29:07,706::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:07,720::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000768027 s, 2.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:07,720::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000768027 s, 2.7 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:29:07,721::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:29:07,721::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:29:07,721::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:29:07,722::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,722::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,722::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,722::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,722::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,723::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:07,829::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:07,831::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:29:07,831::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:29:07,832::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:29:07,832::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:29:07,832::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:29:07,833::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:07,939::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:07,940::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:07,954::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000862038 s, 4.8 MB/s\n'; <rc> = 0
>Thread-1110::DEBUG::2012-12-09 12:29:08,415::task::588::TaskManager.Task::(_updateState) Task=`8335ba9a-7548-4f2e-8e97-d4f64101be0e`::moving from state init -> state preparing
>Thread-1110::INFO::2012-12-09 12:29:08,416::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1110::INFO::2012-12-09 12:29:08,416::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143570899963', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-1110::DEBUG::2012-12-09 12:29:08,417::task::1172::TaskManager.Task::(prepare) Task=`8335ba9a-7548-4f2e-8e97-d4f64101be0e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143570899963', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-1110::DEBUG::2012-12-09 12:29:08,417::task::588::TaskManager.Task::(_updateState) Task=`8335ba9a-7548-4f2e-8e97-d4f64101be0e`::moving from state preparing -> state finished
>Thread-1110::DEBUG::2012-12-09 12:29:08,418::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1110::DEBUG::2012-12-09 12:29:08,418::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1110::DEBUG::2012-12-09 12:29:08,419::task::978::TaskManager.Task::(_decref) Task=`8335ba9a-7548-4f2e-8e97-d4f64101be0e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:08,445::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:08,489::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191947 s, 53.3 MB/s\n'; <rc> = 0
>Thread-1112::DEBUG::2012-12-09 12:29:08,790::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1112::DEBUG::2012-12-09 12:29:08,792::task::588::TaskManager.Task::(_updateState) Task=`37d84654-5a47-4fc2-a5bd-9014b61f08e8`::moving from state init -> state preparing
>Thread-1112::INFO::2012-12-09 12:29:08,792::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1112::INFO::2012-12-09 12:29:08,793::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1112::DEBUG::2012-12-09 12:29:08,793::task::1172::TaskManager.Task::(prepare) Task=`37d84654-5a47-4fc2-a5bd-9014b61f08e8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1112::DEBUG::2012-12-09 12:29:08,794::task::588::TaskManager.Task::(_updateState) Task=`37d84654-5a47-4fc2-a5bd-9014b61f08e8`::moving from state preparing -> state finished
>Thread-1112::DEBUG::2012-12-09 12:29:08,794::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1112::DEBUG::2012-12-09 12:29:08,794::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1112::DEBUG::2012-12-09 12:29:08,795::task::978::TaskManager.Task::(_decref) Task=`37d84654-5a47-4fc2-a5bd-9014b61f08e8`::ref 0 aborting False
>Thread-1113::DEBUG::2012-12-09 12:29:08,815::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1113::DEBUG::2012-12-09 12:29:08,816::task::588::TaskManager.Task::(_updateState) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::moving from state init -> state preparing
>Thread-1113::INFO::2012-12-09 12:29:08,817::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1113::DEBUG::2012-12-09 12:29:08,817::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff0bdb08-d92a-4226-b99b-e40e80833c72`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1113::DEBUG::2012-12-09 12:29:08,818::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1113::DEBUG::2012-12-09 12:29:08,818::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1113::DEBUG::2012-12-09 12:29:08,819::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff0bdb08-d92a-4226-b99b-e40e80833c72`::Granted request
>Thread-1113::DEBUG::2012-12-09 12:29:08,819::task::817::TaskManager.Task::(resourceAcquired) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1113::DEBUG::2012-12-09 12:29:08,820::task::978::TaskManager.Task::(_decref) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::ref 1 aborting False
>Thread-1113::DEBUG::2012-12-09 12:29:08,821::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1113::DEBUG::2012-12-09 12:29:08,822::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1113::DEBUG::2012-12-09 12:29:08,922::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1113::WARNING::2012-12-09 12:29:08,924::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1113::DEBUG::2012-12-09 12:29:08,924::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1113::WARNING::2012-12-09 12:29:08,925::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1113::INFO::2012-12-09 12:29:08,926::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1113::DEBUG::2012-12-09 12:29:08,927::task::1172::TaskManager.Task::(prepare) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1113::DEBUG::2012-12-09 12:29:08,927::task::588::TaskManager.Task::(_updateState) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::moving from state preparing -> state finished
>Thread-1113::DEBUG::2012-12-09 12:29:08,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1113::DEBUG::2012-12-09 12:29:08,928::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1113::DEBUG::2012-12-09 12:29:08,929::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1113::DEBUG::2012-12-09 12:29:08,929::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1113::DEBUG::2012-12-09 12:29:08,929::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1113::DEBUG::2012-12-09 12:29:08,930::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1113::DEBUG::2012-12-09 12:29:08,930::task::978::TaskManager.Task::(_decref) Task=`079aec70-85ba-44f3-ab5e-8880623d32d6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:10,496::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:10,556::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279954 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:12,562::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:12,603::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167184 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:14,610::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:14,661::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237837 s, 43.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:16,668::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:16,710::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180887 s, 56.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:16,803::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:29:16,804::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:16,805::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:29:16,915::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:16,917::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:16,918::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:16,919::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:29:17,957::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:17,969::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000856879 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:18,717::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:18,766::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246043 s, 41.6 MB/s\n'; <rc> = 0
>Thread-1118::DEBUG::2012-12-09 12:29:18,806::task::588::TaskManager.Task::(_updateState) Task=`872bc8a6-f4a2-4474-b1f1-e930146ec7d7`::moving from state init -> state preparing
>Thread-1118::INFO::2012-12-09 12:29:18,807::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1118::INFO::2012-12-09 12:29:18,808::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134320259094', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-1118::DEBUG::2012-12-09 12:29:18,808::task::1172::TaskManager.Task::(prepare) Task=`872bc8a6-f4a2-4474-b1f1-e930146ec7d7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134320259094', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.9', 'code': 358, 'valid': False}}
>Thread-1118::DEBUG::2012-12-09 12:29:18,809::task::588::TaskManager.Task::(_updateState) Task=`872bc8a6-f4a2-4474-b1f1-e930146ec7d7`::moving from state preparing -> state finished
>Thread-1118::DEBUG::2012-12-09 12:29:18,809::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1118::DEBUG::2012-12-09 12:29:18,809::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1118::DEBUG::2012-12-09 12:29:18,810::task::978::TaskManager.Task::(_decref) Task=`872bc8a6-f4a2-4474-b1f1-e930146ec7d7`::ref 0 aborting False
>Thread-1120::DEBUG::2012-12-09 12:29:19,093::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1120::DEBUG::2012-12-09 12:29:19,094::task::588::TaskManager.Task::(_updateState) Task=`71bd6e98-5d49-42a5-ad3a-85ac01d82adf`::moving from state init -> state preparing
>Thread-1120::INFO::2012-12-09 12:29:19,094::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1120::INFO::2012-12-09 12:29:19,095::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1120::DEBUG::2012-12-09 12:29:19,095::task::1172::TaskManager.Task::(prepare) Task=`71bd6e98-5d49-42a5-ad3a-85ac01d82adf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1120::DEBUG::2012-12-09 12:29:19,096::task::588::TaskManager.Task::(_updateState) Task=`71bd6e98-5d49-42a5-ad3a-85ac01d82adf`::moving from state preparing -> state finished
>Thread-1120::DEBUG::2012-12-09 12:29:19,096::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1120::DEBUG::2012-12-09 12:29:19,096::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1120::DEBUG::2012-12-09 12:29:19,097::task::978::TaskManager.Task::(_decref) Task=`71bd6e98-5d49-42a5-ad3a-85ac01d82adf`::ref 0 aborting False
>Thread-1121::DEBUG::2012-12-09 12:29:19,116::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1121::DEBUG::2012-12-09 12:29:19,117::task::588::TaskManager.Task::(_updateState) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::moving from state init -> state preparing
>Thread-1121::INFO::2012-12-09 12:29:19,117::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1121::DEBUG::2012-12-09 12:29:19,117::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`787ecff4-6e35-4e2c-8998-9171ce09a8f1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1121::DEBUG::2012-12-09 12:29:19,118::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1121::DEBUG::2012-12-09 12:29:19,118::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1121::DEBUG::2012-12-09 12:29:19,118::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`787ecff4-6e35-4e2c-8998-9171ce09a8f1`::Granted request
>Thread-1121::DEBUG::2012-12-09 12:29:19,118::task::817::TaskManager.Task::(resourceAcquired) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1121::DEBUG::2012-12-09 12:29:19,119::task::978::TaskManager.Task::(_decref) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::ref 1 aborting False
>Thread-1121::DEBUG::2012-12-09 12:29:19,119::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1121::DEBUG::2012-12-09 12:29:19,120::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1121::DEBUG::2012-12-09 12:29:19,229::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1121::WARNING::2012-12-09 12:29:19,231::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1121::DEBUG::2012-12-09 12:29:19,232::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1121::WARNING::2012-12-09 12:29:19,232::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1121::INFO::2012-12-09 12:29:19,232::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1121::DEBUG::2012-12-09 12:29:19,233::task::1172::TaskManager.Task::(prepare) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1121::DEBUG::2012-12-09 12:29:19,233::task::588::TaskManager.Task::(_updateState) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::moving from state preparing -> state finished
>Thread-1121::DEBUG::2012-12-09 12:29:19,233::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1121::DEBUG::2012-12-09 12:29:19,233::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1121::DEBUG::2012-12-09 12:29:19,233::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1121::DEBUG::2012-12-09 12:29:19,234::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1121::DEBUG::2012-12-09 12:29:19,234::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1121::DEBUG::2012-12-09 12:29:19,234::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1121::DEBUG::2012-12-09 12:29:19,234::task::978::TaskManager.Task::(_decref) Task=`9abf574b-74c2-444d-8fc0-5c601ba7c954`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:20,773::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:20,810::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154308 s, 66.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:22,817::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:22,866::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246931 s, 41.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:24,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:24,924::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272824 s, 37.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:26,920::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:26,921::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:26,931::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:26,971::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017324 s, 59.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:27,031::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:27,032::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:27,032::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:27,033::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:29:27,973::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:28,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00081098 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:28,978::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:29,027::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243321 s, 42.1 MB/s\n'; <rc> = 0
>Thread-1126::DEBUG::2012-12-09 12:29:29,146::task::588::TaskManager.Task::(_updateState) Task=`56668622-67e9-45e3-b4f1-60177c15cd6e`::moving from state init -> state preparing
>Thread-1126::INFO::2012-12-09 12:29:29,147::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1126::INFO::2012-12-09 12:29:29,147::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.027979850769', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.1', 'code': 358, 'valid': False}}
>Thread-1126::DEBUG::2012-12-09 12:29:29,148::task::1172::TaskManager.Task::(prepare) Task=`56668622-67e9-45e3-b4f1-60177c15cd6e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.027979850769', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.1', 'code': 358, 'valid': False}}
>Thread-1126::DEBUG::2012-12-09 12:29:29,148::task::588::TaskManager.Task::(_updateState) Task=`56668622-67e9-45e3-b4f1-60177c15cd6e`::moving from state preparing -> state finished
>Thread-1126::DEBUG::2012-12-09 12:29:29,149::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1126::DEBUG::2012-12-09 12:29:29,149::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1126::DEBUG::2012-12-09 12:29:29,150::task::978::TaskManager.Task::(_decref) Task=`56668622-67e9-45e3-b4f1-60177c15cd6e`::ref 0 aborting False
>Thread-1128::DEBUG::2012-12-09 12:29:29,304::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1128::DEBUG::2012-12-09 12:29:29,305::task::588::TaskManager.Task::(_updateState) Task=`ef500f63-88cc-4f52-9138-510cb0665386`::moving from state init -> state preparing
>Thread-1128::INFO::2012-12-09 12:29:29,305::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1128::INFO::2012-12-09 12:29:29,306::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1128::DEBUG::2012-12-09 12:29:29,306::task::1172::TaskManager.Task::(prepare) Task=`ef500f63-88cc-4f52-9138-510cb0665386`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1128::DEBUG::2012-12-09 12:29:29,307::task::588::TaskManager.Task::(_updateState) Task=`ef500f63-88cc-4f52-9138-510cb0665386`::moving from state preparing -> state finished
>Thread-1128::DEBUG::2012-12-09 12:29:29,307::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1128::DEBUG::2012-12-09 12:29:29,308::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1128::DEBUG::2012-12-09 12:29:29,308::task::978::TaskManager.Task::(_decref) Task=`ef500f63-88cc-4f52-9138-510cb0665386`::ref 0 aborting False
>Thread-1129::DEBUG::2012-12-09 12:29:29,331::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1129::DEBUG::2012-12-09 12:29:29,332::task::588::TaskManager.Task::(_updateState) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::moving from state init -> state preparing
>Thread-1129::INFO::2012-12-09 12:29:29,332::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1129::DEBUG::2012-12-09 12:29:29,333::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8acb1bc0-afea-4409-abb9-5dfc4686a0e6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1129::DEBUG::2012-12-09 12:29:29,334::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1129::DEBUG::2012-12-09 12:29:29,334::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1129::DEBUG::2012-12-09 12:29:29,334::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8acb1bc0-afea-4409-abb9-5dfc4686a0e6`::Granted request
>Thread-1129::DEBUG::2012-12-09 12:29:29,335::task::817::TaskManager.Task::(resourceAcquired) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1129::DEBUG::2012-12-09 12:29:29,336::task::978::TaskManager.Task::(_decref) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::ref 1 aborting False
>Thread-1129::DEBUG::2012-12-09 12:29:29,337::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1129::DEBUG::2012-12-09 12:29:29,338::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1129::DEBUG::2012-12-09 12:29:29,449::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1129::WARNING::2012-12-09 12:29:29,452::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1129::DEBUG::2012-12-09 12:29:29,452::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1129::WARNING::2012-12-09 12:29:29,453::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1129::INFO::2012-12-09 12:29:29,454::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1129::DEBUG::2012-12-09 12:29:29,455::task::1172::TaskManager.Task::(prepare) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1129::DEBUG::2012-12-09 12:29:29,455::task::588::TaskManager.Task::(_updateState) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::moving from state preparing -> state finished
>Thread-1129::DEBUG::2012-12-09 12:29:29,456::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1129::DEBUG::2012-12-09 12:29:29,456::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1129::DEBUG::2012-12-09 12:29:29,456::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1129::DEBUG::2012-12-09 12:29:29,457::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1129::DEBUG::2012-12-09 12:29:29,457::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1129::DEBUG::2012-12-09 12:29:29,457::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1129::DEBUG::2012-12-09 12:29:29,457::task::978::TaskManager.Task::(_decref) Task=`6e95621a-0c07-4d4a-931e-1ed14fa926a6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:31,034::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:31,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199877 s, 51.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:33,085::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:33,128::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205037 s, 49.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:35,135::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:35,186::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027156 s, 37.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:37,034::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:37,035::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:29:37,145::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:37,146::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:37,146::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:37,146::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:29:37,192::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:37,240::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221995 s, 46.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:38,004::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:38,019::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000843681 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:39,247::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:39,298::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254332 s, 40.3 MB/s\n'; <rc> = 0
>Thread-1134::DEBUG::2012-12-09 12:29:39,460::task::588::TaskManager.Task::(_updateState) Task=`45153af0-4efe-4e39-b6b9-f62dc93961aa`::moving from state init -> state preparing
>Thread-1134::INFO::2012-12-09 12:29:39,460::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1134::INFO::2012-12-09 12:29:39,461::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161588191986', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-1134::DEBUG::2012-12-09 12:29:39,461::task::1172::TaskManager.Task::(prepare) Task=`45153af0-4efe-4e39-b6b9-f62dc93961aa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161588191986', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-1134::DEBUG::2012-12-09 12:29:39,462::task::588::TaskManager.Task::(_updateState) Task=`45153af0-4efe-4e39-b6b9-f62dc93961aa`::moving from state preparing -> state finished
>Thread-1134::DEBUG::2012-12-09 12:29:39,462::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1134::DEBUG::2012-12-09 12:29:39,463::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1134::DEBUG::2012-12-09 12:29:39,463::task::978::TaskManager.Task::(_decref) Task=`45153af0-4efe-4e39-b6b9-f62dc93961aa`::ref 0 aborting False
>Thread-1136::DEBUG::2012-12-09 12:29:39,554::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1136::DEBUG::2012-12-09 12:29:39,554::task::588::TaskManager.Task::(_updateState) Task=`62d17c0c-568f-4caf-8b3c-f5f2f630438d`::moving from state init -> state preparing
>Thread-1136::INFO::2012-12-09 12:29:39,554::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1136::INFO::2012-12-09 12:29:39,555::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1136::DEBUG::2012-12-09 12:29:39,555::task::1172::TaskManager.Task::(prepare) Task=`62d17c0c-568f-4caf-8b3c-f5f2f630438d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1136::DEBUG::2012-12-09 12:29:39,555::task::588::TaskManager.Task::(_updateState) Task=`62d17c0c-568f-4caf-8b3c-f5f2f630438d`::moving from state preparing -> state finished
>Thread-1136::DEBUG::2012-12-09 12:29:39,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1136::DEBUG::2012-12-09 12:29:39,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1136::DEBUG::2012-12-09 12:29:39,556::task::978::TaskManager.Task::(_decref) Task=`62d17c0c-568f-4caf-8b3c-f5f2f630438d`::ref 0 aborting False
>Thread-1137::DEBUG::2012-12-09 12:29:39,571::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1137::DEBUG::2012-12-09 12:29:39,572::task::588::TaskManager.Task::(_updateState) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::moving from state init -> state preparing
>Thread-1137::INFO::2012-12-09 12:29:39,572::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1137::DEBUG::2012-12-09 12:29:39,572::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f48fe4a-5f14-45bd-997e-9c96b08d7c09`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1137::DEBUG::2012-12-09 12:29:39,572::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1137::DEBUG::2012-12-09 12:29:39,573::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1137::DEBUG::2012-12-09 12:29:39,573::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f48fe4a-5f14-45bd-997e-9c96b08d7c09`::Granted request
>Thread-1137::DEBUG::2012-12-09 12:29:39,573::task::817::TaskManager.Task::(resourceAcquired) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1137::DEBUG::2012-12-09 12:29:39,573::task::978::TaskManager.Task::(_decref) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::ref 1 aborting False
>Thread-1137::DEBUG::2012-12-09 12:29:39,574::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1137::DEBUG::2012-12-09 12:29:39,574::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1137::DEBUG::2012-12-09 12:29:39,686::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1137::WARNING::2012-12-09 12:29:39,688::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1137::DEBUG::2012-12-09 12:29:39,689::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1137::WARNING::2012-12-09 12:29:39,690::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1137::INFO::2012-12-09 12:29:39,691::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1137::DEBUG::2012-12-09 12:29:39,691::task::1172::TaskManager.Task::(prepare) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1137::DEBUG::2012-12-09 12:29:39,691::task::588::TaskManager.Task::(_updateState) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::moving from state preparing -> state finished
>Thread-1137::DEBUG::2012-12-09 12:29:39,692::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1137::DEBUG::2012-12-09 12:29:39,692::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1137::DEBUG::2012-12-09 12:29:39,692::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1137::DEBUG::2012-12-09 12:29:39,692::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1137::DEBUG::2012-12-09 12:29:39,693::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1137::DEBUG::2012-12-09 12:29:39,693::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1137::DEBUG::2012-12-09 12:29:39,693::task::978::TaskManager.Task::(_decref) Task=`6ac498ff-5b62-4410-bac6-d697f596f2e3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:41,304::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:41,357::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267805 s, 38.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:43,363::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:43,406::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162292 s, 63.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:45,413::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:45,459::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213799 s, 47.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:47,147::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:47,148::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:29:47,262::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:47,263::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:47,263::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:47,263::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:29:47,466::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:47,515::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251814 s, 40.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:48,023::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:48,039::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000956368 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:49,522::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:49,574::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279382 s, 36.7 MB/s\n'; <rc> = 0
>Thread-1142::DEBUG::2012-12-09 12:29:49,769::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1143::DEBUG::2012-12-09 12:29:49,771::task::588::TaskManager.Task::(_updateState) Task=`c0f20757-9972-4e7d-bf37-aebb312b2cb7`::moving from state init -> state preparing
>Thread-1142::DEBUG::2012-12-09 12:29:49,772::task::588::TaskManager.Task::(_updateState) Task=`af5eedc6-2d93-4328-9361-83dd45e9e224`::moving from state init -> state preparing
>Thread-1143::INFO::2012-12-09 12:29:49,773::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1142::INFO::2012-12-09 12:29:49,774::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1143::INFO::2012-12-09 12:29:49,776::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169920921326', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-1142::INFO::2012-12-09 12:29:49,777::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1143::DEBUG::2012-12-09 12:29:49,778::task::1172::TaskManager.Task::(prepare) Task=`c0f20757-9972-4e7d-bf37-aebb312b2cb7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169920921326', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-1142::DEBUG::2012-12-09 12:29:49,779::task::1172::TaskManager.Task::(prepare) Task=`af5eedc6-2d93-4328-9361-83dd45e9e224`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1143::DEBUG::2012-12-09 12:29:49,780::task::588::TaskManager.Task::(_updateState) Task=`c0f20757-9972-4e7d-bf37-aebb312b2cb7`::moving from state preparing -> state finished
>Thread-1142::DEBUG::2012-12-09 12:29:49,781::task::588::TaskManager.Task::(_updateState) Task=`af5eedc6-2d93-4328-9361-83dd45e9e224`::moving from state preparing -> state finished
>Thread-1143::DEBUG::2012-12-09 12:29:49,781::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1142::DEBUG::2012-12-09 12:29:49,782::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1143::DEBUG::2012-12-09 12:29:49,783::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1142::DEBUG::2012-12-09 12:29:49,784::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1143::DEBUG::2012-12-09 12:29:49,785::task::978::TaskManager.Task::(_decref) Task=`c0f20757-9972-4e7d-bf37-aebb312b2cb7`::ref 0 aborting False
>Thread-1142::DEBUG::2012-12-09 12:29:49,786::task::978::TaskManager.Task::(_decref) Task=`af5eedc6-2d93-4328-9361-83dd45e9e224`::ref 0 aborting False
>Thread-1144::DEBUG::2012-12-09 12:29:49,810::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1144::DEBUG::2012-12-09 12:29:49,811::task::588::TaskManager.Task::(_updateState) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::moving from state init -> state preparing
>Thread-1144::INFO::2012-12-09 12:29:49,812::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1144::DEBUG::2012-12-09 12:29:49,813::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f0cce19-adfd-47e7-8aff-54f5cffecf5b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1144::DEBUG::2012-12-09 12:29:49,814::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1144::DEBUG::2012-12-09 12:29:49,814::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1144::DEBUG::2012-12-09 12:29:49,815::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f0cce19-adfd-47e7-8aff-54f5cffecf5b`::Granted request
>Thread-1144::DEBUG::2012-12-09 12:29:49,815::task::817::TaskManager.Task::(resourceAcquired) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1144::DEBUG::2012-12-09 12:29:49,816::task::978::TaskManager.Task::(_decref) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::ref 1 aborting False
>Thread-1144::DEBUG::2012-12-09 12:29:49,817::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1144::DEBUG::2012-12-09 12:29:49,818::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1144::DEBUG::2012-12-09 12:29:49,931::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1144::WARNING::2012-12-09 12:29:49,932::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1144::DEBUG::2012-12-09 12:29:49,932::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1144::WARNING::2012-12-09 12:29:49,933::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1144::INFO::2012-12-09 12:29:49,933::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1144::DEBUG::2012-12-09 12:29:49,934::task::1172::TaskManager.Task::(prepare) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1144::DEBUG::2012-12-09 12:29:49,934::task::588::TaskManager.Task::(_updateState) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::moving from state preparing -> state finished
>Thread-1144::DEBUG::2012-12-09 12:29:49,934::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1144::DEBUG::2012-12-09 12:29:49,935::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1144::DEBUG::2012-12-09 12:29:49,935::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1144::DEBUG::2012-12-09 12:29:49,935::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1144::DEBUG::2012-12-09 12:29:49,935::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1144::DEBUG::2012-12-09 12:29:49,936::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1144::DEBUG::2012-12-09 12:29:49,936::task::978::TaskManager.Task::(_decref) Task=`33379a6f-11db-473c-91ad-d8a457d95f36`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:29:51,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:51,622::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182827 s, 56.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:53,629::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:53,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018809 s, 54.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:55,681::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:55,728::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226681 s, 45.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:29:57,265::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:29:57,266::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:29:57,376::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:29:57,377::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:29:57,378::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:29:57,378::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:29:57,735::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:57,786::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261907 s, 39.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:29:58,043::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:29:58,054::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000420953 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:29:59,792::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:29:59,844::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219433 s, 46.7 MB/s\n'; <rc> = 0
>Thread-1150::DEBUG::2012-12-09 12:30:00,006::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1150::DEBUG::2012-12-09 12:30:00,007::task::588::TaskManager.Task::(_updateState) Task=`f6ba6907-66ff-43ea-8f9e-3a758513d7f3`::moving from state init -> state preparing
>Thread-1150::INFO::2012-12-09 12:30:00,008::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1150::INFO::2012-12-09 12:30:00,008::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1150::DEBUG::2012-12-09 12:30:00,009::task::1172::TaskManager.Task::(prepare) Task=`f6ba6907-66ff-43ea-8f9e-3a758513d7f3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1150::DEBUG::2012-12-09 12:30:00,009::task::588::TaskManager.Task::(_updateState) Task=`f6ba6907-66ff-43ea-8f9e-3a758513d7f3`::moving from state preparing -> state finished
>Thread-1150::DEBUG::2012-12-09 12:30:00,009::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1150::DEBUG::2012-12-09 12:30:00,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1150::DEBUG::2012-12-09 12:30:00,010::task::978::TaskManager.Task::(_decref) Task=`f6ba6907-66ff-43ea-8f9e-3a758513d7f3`::ref 0 aborting False
>Thread-1151::DEBUG::2012-12-09 12:30:00,034::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1151::DEBUG::2012-12-09 12:30:00,035::task::588::TaskManager.Task::(_updateState) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::moving from state init -> state preparing
>Thread-1151::INFO::2012-12-09 12:30:00,035::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1151::DEBUG::2012-12-09 12:30:00,036::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a03b07b3-3043-47ae-8f73-434127b975b5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1151::DEBUG::2012-12-09 12:30:00,036::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1151::DEBUG::2012-12-09 12:30:00,037::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1151::DEBUG::2012-12-09 12:30:00,037::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a03b07b3-3043-47ae-8f73-434127b975b5`::Granted request
>Thread-1151::DEBUG::2012-12-09 12:30:00,038::task::817::TaskManager.Task::(resourceAcquired) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1151::DEBUG::2012-12-09 12:30:00,038::task::978::TaskManager.Task::(_decref) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::ref 1 aborting False
>Thread-1151::DEBUG::2012-12-09 12:30:00,040::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1151::DEBUG::2012-12-09 12:30:00,041::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1152::DEBUG::2012-12-09 12:30:00,086::task::588::TaskManager.Task::(_updateState) Task=`a0ef0061-0aa9-412a-85df-46ab79c18194`::moving from state init -> state preparing
>Thread-1152::INFO::2012-12-09 12:30:00,086::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1152::INFO::2012-12-09 12:30:00,087::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125451087952', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1152::DEBUG::2012-12-09 12:30:00,087::task::1172::TaskManager.Task::(prepare) Task=`a0ef0061-0aa9-412a-85df-46ab79c18194`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125451087952', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1152::DEBUG::2012-12-09 12:30:00,087::task::588::TaskManager.Task::(_updateState) Task=`a0ef0061-0aa9-412a-85df-46ab79c18194`::moving from state preparing -> state finished
>Thread-1152::DEBUG::2012-12-09 12:30:00,087::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1152::DEBUG::2012-12-09 12:30:00,088::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1152::DEBUG::2012-12-09 12:30:00,088::task::978::TaskManager.Task::(_decref) Task=`a0ef0061-0aa9-412a-85df-46ab79c18194`::ref 0 aborting False
>Thread-1151::DEBUG::2012-12-09 12:30:00,157::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1151::WARNING::2012-12-09 12:30:00,158::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1151::DEBUG::2012-12-09 12:30:00,158::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1151::WARNING::2012-12-09 12:30:00,158::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1151::INFO::2012-12-09 12:30:00,159::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1151::DEBUG::2012-12-09 12:30:00,159::task::1172::TaskManager.Task::(prepare) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1151::DEBUG::2012-12-09 12:30:00,159::task::588::TaskManager.Task::(_updateState) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::moving from state preparing -> state finished
>Thread-1151::DEBUG::2012-12-09 12:30:00,159::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1151::DEBUG::2012-12-09 12:30:00,160::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1151::DEBUG::2012-12-09 12:30:00,160::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1151::DEBUG::2012-12-09 12:30:00,160::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1151::DEBUG::2012-12-09 12:30:00,160::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1151::DEBUG::2012-12-09 12:30:00,161::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1151::DEBUG::2012-12-09 12:30:00,161::task::978::TaskManager.Task::(_decref) Task=`96746653-63a9-4011-a7f0-fbe7fd6bb36a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:01,850::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:01,892::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182721 s, 56.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:03,899::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:03,947::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238607 s, 42.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:05,954::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:06,000::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216391 s, 47.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:07,380::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:07,381::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:07,491::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:07,492::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:07,492::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:07,493::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:30:08,006::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:08,049::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186087 s, 55.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:30:08,059::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:08,074::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0008572 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:10,056::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:10,106::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250726 s, 40.8 MB/s\n'; <rc> = 0
>Thread-1158::DEBUG::2012-12-09 12:30:10,385::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1158::DEBUG::2012-12-09 12:30:10,386::task::588::TaskManager.Task::(_updateState) Task=`18ed26a9-7a6f-4827-b8a2-c795231a6174`::moving from state init -> state preparing
>Thread-1158::INFO::2012-12-09 12:30:10,387::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1158::INFO::2012-12-09 12:30:10,387::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1158::DEBUG::2012-12-09 12:30:10,388::task::1172::TaskManager.Task::(prepare) Task=`18ed26a9-7a6f-4827-b8a2-c795231a6174`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1158::DEBUG::2012-12-09 12:30:10,388::task::588::TaskManager.Task::(_updateState) Task=`18ed26a9-7a6f-4827-b8a2-c795231a6174`::moving from state preparing -> state finished
>Thread-1158::DEBUG::2012-12-09 12:30:10,389::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1158::DEBUG::2012-12-09 12:30:10,389::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1158::DEBUG::2012-12-09 12:30:10,390::task::978::TaskManager.Task::(_decref) Task=`18ed26a9-7a6f-4827-b8a2-c795231a6174`::ref 0 aborting False
>Thread-1159::DEBUG::2012-12-09 12:30:10,413::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1159::DEBUG::2012-12-09 12:30:10,414::task::588::TaskManager.Task::(_updateState) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::moving from state init -> state preparing
>Thread-1159::INFO::2012-12-09 12:30:10,414::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1159::DEBUG::2012-12-09 12:30:10,415::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`09ee5f23-43bc-48d2-9575-06186b4d9e59`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1159::DEBUG::2012-12-09 12:30:10,415::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1159::DEBUG::2012-12-09 12:30:10,416::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1159::DEBUG::2012-12-09 12:30:10,416::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`09ee5f23-43bc-48d2-9575-06186b4d9e59`::Granted request
>Thread-1159::DEBUG::2012-12-09 12:30:10,417::task::817::TaskManager.Task::(resourceAcquired) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1159::DEBUG::2012-12-09 12:30:10,417::task::978::TaskManager.Task::(_decref) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::ref 1 aborting False
>Thread-1159::DEBUG::2012-12-09 12:30:10,419::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1159::DEBUG::2012-12-09 12:30:10,420::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1160::DEBUG::2012-12-09 12:30:10,494::task::588::TaskManager.Task::(_updateState) Task=`63581556-0fb4-4126-94be-a3c9d426cd66`::moving from state init -> state preparing
>Thread-1160::INFO::2012-12-09 12:30:10,494::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1160::INFO::2012-12-09 12:30:10,494::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161159038544', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-1160::DEBUG::2012-12-09 12:30:10,495::task::1172::TaskManager.Task::(prepare) Task=`63581556-0fb4-4126-94be-a3c9d426cd66`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161159038544', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-1160::DEBUG::2012-12-09 12:30:10,495::task::588::TaskManager.Task::(_updateState) Task=`63581556-0fb4-4126-94be-a3c9d426cd66`::moving from state preparing -> state finished
>Thread-1160::DEBUG::2012-12-09 12:30:10,495::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1160::DEBUG::2012-12-09 12:30:10,496::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1160::DEBUG::2012-12-09 12:30:10,496::task::978::TaskManager.Task::(_decref) Task=`63581556-0fb4-4126-94be-a3c9d426cd66`::ref 0 aborting False
>Thread-1159::DEBUG::2012-12-09 12:30:10,524::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1159::WARNING::2012-12-09 12:30:10,525::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1159::DEBUG::2012-12-09 12:30:10,526::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1159::WARNING::2012-12-09 12:30:10,527::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1159::INFO::2012-12-09 12:30:10,527::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1159::DEBUG::2012-12-09 12:30:10,527::task::1172::TaskManager.Task::(prepare) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1159::DEBUG::2012-12-09 12:30:10,528::task::588::TaskManager.Task::(_updateState) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::moving from state preparing -> state finished
>Thread-1159::DEBUG::2012-12-09 12:30:10,528::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1159::DEBUG::2012-12-09 12:30:10,528::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1159::DEBUG::2012-12-09 12:30:10,529::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1159::DEBUG::2012-12-09 12:30:10,529::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1159::DEBUG::2012-12-09 12:30:10,529::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1159::DEBUG::2012-12-09 12:30:10,530::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1159::DEBUG::2012-12-09 12:30:10,530::task::978::TaskManager.Task::(_decref) Task=`bdba15ea-c874-43ea-9d87-0d28ceebe74c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:12,112::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:12,151::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170915 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:14,157::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:14,195::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163636 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:16,201::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:16,253::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278661 s, 36.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:17,494::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:17,494::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:17,592::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:17,594::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:17,594::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:17,594::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:30:18,080::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:18,090::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000583159 s, 7.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:18,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:18,314::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283332 s, 36.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:20,320::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:20,368::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236093 s, 43.4 MB/s\n'; <rc> = 0
>Thread-1166::DEBUG::2012-12-09 12:30:20,677::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1166::DEBUG::2012-12-09 12:30:20,678::task::588::TaskManager.Task::(_updateState) Task=`badb6e7d-01bf-4735-8329-ec797ae22a47`::moving from state init -> state preparing
>Thread-1166::INFO::2012-12-09 12:30:20,679::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1166::INFO::2012-12-09 12:30:20,679::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1166::DEBUG::2012-12-09 12:30:20,680::task::1172::TaskManager.Task::(prepare) Task=`badb6e7d-01bf-4735-8329-ec797ae22a47`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1166::DEBUG::2012-12-09 12:30:20,680::task::588::TaskManager.Task::(_updateState) Task=`badb6e7d-01bf-4735-8329-ec797ae22a47`::moving from state preparing -> state finished
>Thread-1166::DEBUG::2012-12-09 12:30:20,681::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1166::DEBUG::2012-12-09 12:30:20,681::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1166::DEBUG::2012-12-09 12:30:20,682::task::978::TaskManager.Task::(_decref) Task=`badb6e7d-01bf-4735-8329-ec797ae22a47`::ref 0 aborting False
>Thread-1167::DEBUG::2012-12-09 12:30:20,704::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1167::DEBUG::2012-12-09 12:30:20,704::task::588::TaskManager.Task::(_updateState) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::moving from state init -> state preparing
>Thread-1167::INFO::2012-12-09 12:30:20,705::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1167::DEBUG::2012-12-09 12:30:20,706::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f87db3c8-472b-4fb8-a481-5e4cd01c1149`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1167::DEBUG::2012-12-09 12:30:20,706::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1167::DEBUG::2012-12-09 12:30:20,707::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1167::DEBUG::2012-12-09 12:30:20,707::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f87db3c8-472b-4fb8-a481-5e4cd01c1149`::Granted request
>Thread-1167::DEBUG::2012-12-09 12:30:20,708::task::817::TaskManager.Task::(resourceAcquired) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1167::DEBUG::2012-12-09 12:30:20,708::task::978::TaskManager.Task::(_decref) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::ref 1 aborting False
>Thread-1167::DEBUG::2012-12-09 12:30:20,709::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1167::DEBUG::2012-12-09 12:30:20,710::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1168::DEBUG::2012-12-09 12:30:20,788::task::588::TaskManager.Task::(_updateState) Task=`de7561a9-b544-4a7e-8377-f119481c0175`::moving from state init -> state preparing
>Thread-1168::INFO::2012-12-09 12:30:20,788::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1168::INFO::2012-12-09 12:30:20,788::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103750228882', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-1168::DEBUG::2012-12-09 12:30:20,789::task::1172::TaskManager.Task::(prepare) Task=`de7561a9-b544-4a7e-8377-f119481c0175`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103750228882', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-1168::DEBUG::2012-12-09 12:30:20,789::task::588::TaskManager.Task::(_updateState) Task=`de7561a9-b544-4a7e-8377-f119481c0175`::moving from state preparing -> state finished
>Thread-1168::DEBUG::2012-12-09 12:30:20,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1168::DEBUG::2012-12-09 12:30:20,789::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1168::DEBUG::2012-12-09 12:30:20,790::task::978::TaskManager.Task::(_decref) Task=`de7561a9-b544-4a7e-8377-f119481c0175`::ref 0 aborting False
>Thread-1167::DEBUG::2012-12-09 12:30:20,817::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1167::WARNING::2012-12-09 12:30:20,819::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1167::DEBUG::2012-12-09 12:30:20,819::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1167::WARNING::2012-12-09 12:30:20,820::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1167::INFO::2012-12-09 12:30:20,821::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1167::DEBUG::2012-12-09 12:30:20,821::task::1172::TaskManager.Task::(prepare) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1167::DEBUG::2012-12-09 12:30:20,822::task::588::TaskManager.Task::(_updateState) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::moving from state preparing -> state finished
>Thread-1167::DEBUG::2012-12-09 12:30:20,822::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1167::DEBUG::2012-12-09 12:30:20,822::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1167::DEBUG::2012-12-09 12:30:20,822::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1167::DEBUG::2012-12-09 12:30:20,822::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1167::DEBUG::2012-12-09 12:30:20,823::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1167::DEBUG::2012-12-09 12:30:20,823::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1167::DEBUG::2012-12-09 12:30:20,823::task::978::TaskManager.Task::(_decref) Task=`7c33420a-1a74-4523-8791-42df64ac2be1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:22,374::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:22,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210201 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:24,427::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:24,477::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241798 s, 42.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:26,483::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:26,521::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158367 s, 64.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:27,595::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:27,596::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:27,707::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:27,708::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:27,709::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:27,709::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:30:28,094::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:28,109::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000907629 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:28,528::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:28,576::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023896 s, 42.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:30,582::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:30,634::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221683 s, 46.2 MB/s\n'; <rc> = 0
>Thread-1174::DEBUG::2012-12-09 12:30:30,953::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1174::DEBUG::2012-12-09 12:30:30,954::task::588::TaskManager.Task::(_updateState) Task=`08cc4b43-49cb-42ac-b174-b65e84fa58d2`::moving from state init -> state preparing
>Thread-1174::INFO::2012-12-09 12:30:30,955::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1174::INFO::2012-12-09 12:30:30,955::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1174::DEBUG::2012-12-09 12:30:30,956::task::1172::TaskManager.Task::(prepare) Task=`08cc4b43-49cb-42ac-b174-b65e84fa58d2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1174::DEBUG::2012-12-09 12:30:30,956::task::588::TaskManager.Task::(_updateState) Task=`08cc4b43-49cb-42ac-b174-b65e84fa58d2`::moving from state preparing -> state finished
>Thread-1174::DEBUG::2012-12-09 12:30:30,956::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1174::DEBUG::2012-12-09 12:30:30,957::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1174::DEBUG::2012-12-09 12:30:30,957::task::978::TaskManager.Task::(_decref) Task=`08cc4b43-49cb-42ac-b174-b65e84fa58d2`::ref 0 aborting False
>Thread-1175::DEBUG::2012-12-09 12:30:30,977::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1175::DEBUG::2012-12-09 12:30:30,978::task::588::TaskManager.Task::(_updateState) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::moving from state init -> state preparing
>Thread-1175::INFO::2012-12-09 12:30:30,979::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1175::DEBUG::2012-12-09 12:30:30,980::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d96a870c-d6bc-4284-bfe0-21b3a33258dc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1175::DEBUG::2012-12-09 12:30:30,980::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1175::DEBUG::2012-12-09 12:30:30,980::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1175::DEBUG::2012-12-09 12:30:30,981::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d96a870c-d6bc-4284-bfe0-21b3a33258dc`::Granted request
>Thread-1175::DEBUG::2012-12-09 12:30:30,981::task::817::TaskManager.Task::(resourceAcquired) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1175::DEBUG::2012-12-09 12:30:30,982::task::978::TaskManager.Task::(_decref) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::ref 1 aborting False
>Thread-1175::DEBUG::2012-12-09 12:30:30,982::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1175::DEBUG::2012-12-09 12:30:30,983::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1176::DEBUG::2012-12-09 12:30:31,084::task::588::TaskManager.Task::(_updateState) Task=`ab1de6cf-e44a-4d42-a17f-d93e270af3e7`::moving from state init -> state preparing
>Thread-1176::INFO::2012-12-09 12:30:31,084::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1176::INFO::2012-12-09 12:30:31,085::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0165069103241', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-1176::DEBUG::2012-12-09 12:30:31,085::task::1172::TaskManager.Task::(prepare) Task=`ab1de6cf-e44a-4d42-a17f-d93e270af3e7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0165069103241', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-1176::DEBUG::2012-12-09 12:30:31,085::task::588::TaskManager.Task::(_updateState) Task=`ab1de6cf-e44a-4d42-a17f-d93e270af3e7`::moving from state preparing -> state finished
>Thread-1176::DEBUG::2012-12-09 12:30:31,085::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1176::DEBUG::2012-12-09 12:30:31,085::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1176::DEBUG::2012-12-09 12:30:31,086::task::978::TaskManager.Task::(_decref) Task=`ab1de6cf-e44a-4d42-a17f-d93e270af3e7`::ref 0 aborting False
>Thread-1175::DEBUG::2012-12-09 12:30:31,090::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1175::WARNING::2012-12-09 12:30:31,092::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1175::DEBUG::2012-12-09 12:30:31,092::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1175::WARNING::2012-12-09 12:30:31,092::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1175::INFO::2012-12-09 12:30:31,093::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1175::DEBUG::2012-12-09 12:30:31,093::task::1172::TaskManager.Task::(prepare) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1175::DEBUG::2012-12-09 12:30:31,093::task::588::TaskManager.Task::(_updateState) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::moving from state preparing -> state finished
>Thread-1175::DEBUG::2012-12-09 12:30:31,093::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1175::DEBUG::2012-12-09 12:30:31,093::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1175::DEBUG::2012-12-09 12:30:31,094::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1175::DEBUG::2012-12-09 12:30:31,094::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1175::DEBUG::2012-12-09 12:30:31,094::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1175::DEBUG::2012-12-09 12:30:31,094::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1175::DEBUG::2012-12-09 12:30:31,095::task::978::TaskManager.Task::(_decref) Task=`37ea6d2c-ab78-48c7-a146-2584831eb108`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:32,641::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:32,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312798 s, 32.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:34,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:34,756::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251356 s, 40.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:36,762::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:36,815::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282876 s, 36.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:37,711::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:37,711::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:37,813::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:37,814::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:37,814::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:37,815::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:30:38,113::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:38,122::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000440139 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:38,822::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:38,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216762 s, 47.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:40,880::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:40,922::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194158 s, 52.7 MB/s\n'; <rc> = 0
>Thread-1182::DEBUG::2012-12-09 12:30:41,248::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1182::DEBUG::2012-12-09 12:30:41,250::task::588::TaskManager.Task::(_updateState) Task=`b24c5b08-2a69-4080-a4cf-48ba371e1312`::moving from state init -> state preparing
>Thread-1182::INFO::2012-12-09 12:30:41,251::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1182::INFO::2012-12-09 12:30:41,251::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1182::DEBUG::2012-12-09 12:30:41,251::task::1172::TaskManager.Task::(prepare) Task=`b24c5b08-2a69-4080-a4cf-48ba371e1312`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1182::DEBUG::2012-12-09 12:30:41,252::task::588::TaskManager.Task::(_updateState) Task=`b24c5b08-2a69-4080-a4cf-48ba371e1312`::moving from state preparing -> state finished
>Thread-1182::DEBUG::2012-12-09 12:30:41,252::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1182::DEBUG::2012-12-09 12:30:41,252::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1182::DEBUG::2012-12-09 12:30:41,252::task::978::TaskManager.Task::(_decref) Task=`b24c5b08-2a69-4080-a4cf-48ba371e1312`::ref 0 aborting False
>Thread-1183::DEBUG::2012-12-09 12:30:41,269::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1183::DEBUG::2012-12-09 12:30:41,270::task::588::TaskManager.Task::(_updateState) Task=`90834e95-50bf-4968-b180-38a814a0e654`::moving from state init -> state preparing
>Thread-1183::INFO::2012-12-09 12:30:41,271::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1183::DEBUG::2012-12-09 12:30:41,272::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`23de319e-750b-4677-824d-7f7c9622e0ec`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1183::DEBUG::2012-12-09 12:30:41,272::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1183::DEBUG::2012-12-09 12:30:41,272::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1183::DEBUG::2012-12-09 12:30:41,273::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`23de319e-750b-4677-824d-7f7c9622e0ec`::Granted request
>Thread-1183::DEBUG::2012-12-09 12:30:41,274::task::817::TaskManager.Task::(resourceAcquired) Task=`90834e95-50bf-4968-b180-38a814a0e654`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1183::DEBUG::2012-12-09 12:30:41,274::task::978::TaskManager.Task::(_decref) Task=`90834e95-50bf-4968-b180-38a814a0e654`::ref 1 aborting False
>Thread-1183::DEBUG::2012-12-09 12:30:41,276::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1183::DEBUG::2012-12-09 12:30:41,277::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1183::DEBUG::2012-12-09 12:30:41,396::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1183::WARNING::2012-12-09 12:30:41,398::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1183::DEBUG::2012-12-09 12:30:41,399::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1183::WARNING::2012-12-09 12:30:41,400::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1183::INFO::2012-12-09 12:30:41,400::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1183::DEBUG::2012-12-09 12:30:41,401::task::1172::TaskManager.Task::(prepare) Task=`90834e95-50bf-4968-b180-38a814a0e654`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1183::DEBUG::2012-12-09 12:30:41,401::task::588::TaskManager.Task::(_updateState) Task=`90834e95-50bf-4968-b180-38a814a0e654`::moving from state preparing -> state finished
>Thread-1183::DEBUG::2012-12-09 12:30:41,402::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1183::DEBUG::2012-12-09 12:30:41,402::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1183::DEBUG::2012-12-09 12:30:41,402::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1183::DEBUG::2012-12-09 12:30:41,402::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1183::DEBUG::2012-12-09 12:30:41,402::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1183::DEBUG::2012-12-09 12:30:41,403::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1183::DEBUG::2012-12-09 12:30:41,403::task::978::TaskManager.Task::(_decref) Task=`90834e95-50bf-4968-b180-38a814a0e654`::ref 0 aborting False
>Thread-1184::DEBUG::2012-12-09 12:30:41,496::task::588::TaskManager.Task::(_updateState) Task=`65a4e14b-bd07-4b09-a159-3191da594ee5`::moving from state init -> state preparing
>Thread-1184::INFO::2012-12-09 12:30:41,497::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1184::INFO::2012-12-09 12:30:41,497::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00944018363953', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.7', 'code': 358, 'valid': False}}
>Thread-1184::DEBUG::2012-12-09 12:30:41,498::task::1172::TaskManager.Task::(prepare) Task=`65a4e14b-bd07-4b09-a159-3191da594ee5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00944018363953', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.7', 'code': 358, 'valid': False}}
>Thread-1184::DEBUG::2012-12-09 12:30:41,498::task::588::TaskManager.Task::(_updateState) Task=`65a4e14b-bd07-4b09-a159-3191da594ee5`::moving from state preparing -> state finished
>Thread-1184::DEBUG::2012-12-09 12:30:41,499::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1184::DEBUG::2012-12-09 12:30:41,499::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1184::DEBUG::2012-12-09 12:30:41,500::task::978::TaskManager.Task::(_decref) Task=`65a4e14b-bd07-4b09-a159-3191da594ee5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:42,929::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:42,977::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233441 s, 43.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:44,983::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:45,035::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275172 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:47,042::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:47,089::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220064 s, 46.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:47,816::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:47,816::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:47,916::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:47,918::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:47,918::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:47,919::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:30:48,125::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:48,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000991755 s, 4.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:49,095::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:49,145::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257546 s, 39.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:51,152::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:51,193::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177001 s, 57.9 MB/s\n'; <rc> = 0
>Thread-1190::DEBUG::2012-12-09 12:30:51,576::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1190::DEBUG::2012-12-09 12:30:51,577::task::588::TaskManager.Task::(_updateState) Task=`26aacc89-46b1-4587-a82d-d146a27f018c`::moving from state init -> state preparing
>Thread-1190::INFO::2012-12-09 12:30:51,577::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1190::INFO::2012-12-09 12:30:51,577::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1190::DEBUG::2012-12-09 12:30:51,577::task::1172::TaskManager.Task::(prepare) Task=`26aacc89-46b1-4587-a82d-d146a27f018c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1190::DEBUG::2012-12-09 12:30:51,578::task::588::TaskManager.Task::(_updateState) Task=`26aacc89-46b1-4587-a82d-d146a27f018c`::moving from state preparing -> state finished
>Thread-1190::DEBUG::2012-12-09 12:30:51,578::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1190::DEBUG::2012-12-09 12:30:51,578::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1190::DEBUG::2012-12-09 12:30:51,578::task::978::TaskManager.Task::(_decref) Task=`26aacc89-46b1-4587-a82d-d146a27f018c`::ref 0 aborting False
>Thread-1191::DEBUG::2012-12-09 12:30:51,590::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1191::DEBUG::2012-12-09 12:30:51,591::task::588::TaskManager.Task::(_updateState) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::moving from state init -> state preparing
>Thread-1191::INFO::2012-12-09 12:30:51,591::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1191::DEBUG::2012-12-09 12:30:51,591::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2902bc9f-6962-42d0-9451-8c1fd20dba99`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1191::DEBUG::2012-12-09 12:30:51,591::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1191::DEBUG::2012-12-09 12:30:51,592::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1191::DEBUG::2012-12-09 12:30:51,592::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2902bc9f-6962-42d0-9451-8c1fd20dba99`::Granted request
>Thread-1191::DEBUG::2012-12-09 12:30:51,592::task::817::TaskManager.Task::(resourceAcquired) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1191::DEBUG::2012-12-09 12:30:51,592::task::978::TaskManager.Task::(_decref) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::ref 1 aborting False
>Thread-1191::DEBUG::2012-12-09 12:30:51,593::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1191::DEBUG::2012-12-09 12:30:51,593::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1192::DEBUG::2012-12-09 12:30:51,691::task::588::TaskManager.Task::(_updateState) Task=`96dd1b4f-03c9-4ab4-a5bd-869ad4fc7332`::moving from state init -> state preparing
>Thread-1192::INFO::2012-12-09 12:30:51,691::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1192::INFO::2012-12-09 12:30:51,692::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160069465637', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1192::DEBUG::2012-12-09 12:30:51,692::task::1172::TaskManager.Task::(prepare) Task=`96dd1b4f-03c9-4ab4-a5bd-869ad4fc7332`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160069465637', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1192::DEBUG::2012-12-09 12:30:51,692::task::588::TaskManager.Task::(_updateState) Task=`96dd1b4f-03c9-4ab4-a5bd-869ad4fc7332`::moving from state preparing -> state finished
>Thread-1192::DEBUG::2012-12-09 12:30:51,692::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1192::DEBUG::2012-12-09 12:30:51,693::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1192::DEBUG::2012-12-09 12:30:51,693::task::978::TaskManager.Task::(_decref) Task=`96dd1b4f-03c9-4ab4-a5bd-869ad4fc7332`::ref 0 aborting False
>Thread-1191::DEBUG::2012-12-09 12:30:51,697::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1191::WARNING::2012-12-09 12:30:51,699::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1191::DEBUG::2012-12-09 12:30:51,700::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1191::WARNING::2012-12-09 12:30:51,701::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1191::INFO::2012-12-09 12:30:51,702::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1191::DEBUG::2012-12-09 12:30:51,703::task::1172::TaskManager.Task::(prepare) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1191::DEBUG::2012-12-09 12:30:51,703::task::588::TaskManager.Task::(_updateState) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::moving from state preparing -> state finished
>Thread-1191::DEBUG::2012-12-09 12:30:51,703::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1191::DEBUG::2012-12-09 12:30:51,704::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1191::DEBUG::2012-12-09 12:30:51,704::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1191::DEBUG::2012-12-09 12:30:51,704::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1191::DEBUG::2012-12-09 12:30:51,705::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1191::DEBUG::2012-12-09 12:30:51,705::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1191::DEBUG::2012-12-09 12:30:51,705::task::978::TaskManager.Task::(_decref) Task=`ea06928b-570c-48f5-8039-f2f705f832bd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:30:53,200::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:53,243::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198019 s, 51.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:55,249::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:55,288::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163861 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:57,295::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:57,334::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156991 s, 65.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:30:57,920::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:30:57,921::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:30:58,016::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:30:58,017::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:30:58,017::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:30:58,018::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:30:58,146::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:30:58,157::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000515592 s, 7.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:30:59,341::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:30:59,387::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222066 s, 46.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:01,392::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:01,445::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276761 s, 37.0 MB/s\n'; <rc> = 0
>Thread-1198::DEBUG::2012-12-09 12:31:01,885::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1198::DEBUG::2012-12-09 12:31:01,886::task::588::TaskManager.Task::(_updateState) Task=`25618a5a-0c0b-4ce0-bb45-e0df33bf49e9`::moving from state init -> state preparing
>Thread-1198::INFO::2012-12-09 12:31:01,886::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1198::INFO::2012-12-09 12:31:01,887::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1198::DEBUG::2012-12-09 12:31:01,887::task::1172::TaskManager.Task::(prepare) Task=`25618a5a-0c0b-4ce0-bb45-e0df33bf49e9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1198::DEBUG::2012-12-09 12:31:01,888::task::588::TaskManager.Task::(_updateState) Task=`25618a5a-0c0b-4ce0-bb45-e0df33bf49e9`::moving from state preparing -> state finished
>Thread-1198::DEBUG::2012-12-09 12:31:01,888::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1198::DEBUG::2012-12-09 12:31:01,888::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1198::DEBUG::2012-12-09 12:31:01,889::task::978::TaskManager.Task::(_decref) Task=`25618a5a-0c0b-4ce0-bb45-e0df33bf49e9`::ref 0 aborting False
>Thread-1199::DEBUG::2012-12-09 12:31:01,913::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1199::DEBUG::2012-12-09 12:31:01,914::task::588::TaskManager.Task::(_updateState) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::moving from state init -> state preparing
>Thread-1199::INFO::2012-12-09 12:31:01,914::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1199::DEBUG::2012-12-09 12:31:01,915::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f0c714f1-67a3-41bc-b4ea-5a77cc0a9fd4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1199::DEBUG::2012-12-09 12:31:01,916::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1199::DEBUG::2012-12-09 12:31:01,916::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1199::DEBUG::2012-12-09 12:31:01,917::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f0c714f1-67a3-41bc-b4ea-5a77cc0a9fd4`::Granted request
>Thread-1199::DEBUG::2012-12-09 12:31:01,917::task::817::TaskManager.Task::(resourceAcquired) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1199::DEBUG::2012-12-09 12:31:01,918::task::978::TaskManager.Task::(_decref) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::ref 1 aborting False
>Thread-1199::DEBUG::2012-12-09 12:31:01,919::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1199::DEBUG::2012-12-09 12:31:01,920::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1200::DEBUG::2012-12-09 12:31:01,979::task::588::TaskManager.Task::(_updateState) Task=`947c3eb3-ae13-4f7f-9d8f-baccd60de7a8`::moving from state init -> state preparing
>Thread-1200::INFO::2012-12-09 12:31:01,979::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1200::INFO::2012-12-09 12:31:01,980::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112671852112', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-1200::DEBUG::2012-12-09 12:31:01,980::task::1172::TaskManager.Task::(prepare) Task=`947c3eb3-ae13-4f7f-9d8f-baccd60de7a8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112671852112', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-1200::DEBUG::2012-12-09 12:31:01,980::task::588::TaskManager.Task::(_updateState) Task=`947c3eb3-ae13-4f7f-9d8f-baccd60de7a8`::moving from state preparing -> state finished
>Thread-1200::DEBUG::2012-12-09 12:31:01,980::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1200::DEBUG::2012-12-09 12:31:01,981::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1200::DEBUG::2012-12-09 12:31:01,981::task::978::TaskManager.Task::(_decref) Task=`947c3eb3-ae13-4f7f-9d8f-baccd60de7a8`::ref 0 aborting False
>Thread-1199::DEBUG::2012-12-09 12:31:02,030::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1199::WARNING::2012-12-09 12:31:02,031::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1199::DEBUG::2012-12-09 12:31:02,032::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1199::WARNING::2012-12-09 12:31:02,032::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1199::INFO::2012-12-09 12:31:02,032::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1199::DEBUG::2012-12-09 12:31:02,033::task::1172::TaskManager.Task::(prepare) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1199::DEBUG::2012-12-09 12:31:02,033::task::588::TaskManager.Task::(_updateState) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::moving from state preparing -> state finished
>Thread-1199::DEBUG::2012-12-09 12:31:02,033::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1199::DEBUG::2012-12-09 12:31:02,033::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1199::DEBUG::2012-12-09 12:31:02,033::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1199::DEBUG::2012-12-09 12:31:02,034::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1199::DEBUG::2012-12-09 12:31:02,034::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1199::DEBUG::2012-12-09 12:31:02,034::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1199::DEBUG::2012-12-09 12:31:02,034::task::978::TaskManager.Task::(_decref) Task=`d7d79643-2a4a-413b-9ccb-908ac74a48cf`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:03,452::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:03,495::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169535 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:05,501::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:05,554::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279464 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:07,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:07,607::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234737 s, 43.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:08,019::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:08,020::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:31:08,132::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:08,133::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:08,133::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:08,134::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:31:08,160::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:08,171::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000757745 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:09,614::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:09,657::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189958 s, 53.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:11,664::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:11,716::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257549 s, 39.8 MB/s\n'; <rc> = 0
>Thread-1206::DEBUG::2012-12-09 12:31:12,115::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1206::DEBUG::2012-12-09 12:31:12,116::task::588::TaskManager.Task::(_updateState) Task=`7ef11908-292c-4ac7-aba8-1e1422efe754`::moving from state init -> state preparing
>Thread-1206::INFO::2012-12-09 12:31:12,116::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1206::INFO::2012-12-09 12:31:12,117::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1206::DEBUG::2012-12-09 12:31:12,118::task::1172::TaskManager.Task::(prepare) Task=`7ef11908-292c-4ac7-aba8-1e1422efe754`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1206::DEBUG::2012-12-09 12:31:12,118::task::588::TaskManager.Task::(_updateState) Task=`7ef11908-292c-4ac7-aba8-1e1422efe754`::moving from state preparing -> state finished
>Thread-1206::DEBUG::2012-12-09 12:31:12,118::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1206::DEBUG::2012-12-09 12:31:12,119::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1206::DEBUG::2012-12-09 12:31:12,119::task::978::TaskManager.Task::(_decref) Task=`7ef11908-292c-4ac7-aba8-1e1422efe754`::ref 0 aborting False
>Thread-1207::DEBUG::2012-12-09 12:31:12,143::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1207::DEBUG::2012-12-09 12:31:12,144::task::588::TaskManager.Task::(_updateState) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::moving from state init -> state preparing
>Thread-1207::INFO::2012-12-09 12:31:12,144::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1207::DEBUG::2012-12-09 12:31:12,145::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a1dde17d-7e26-4b0f-bcd7-eaaf9c44a41a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1207::DEBUG::2012-12-09 12:31:12,146::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1207::DEBUG::2012-12-09 12:31:12,146::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1207::DEBUG::2012-12-09 12:31:12,147::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a1dde17d-7e26-4b0f-bcd7-eaaf9c44a41a`::Granted request
>Thread-1207::DEBUG::2012-12-09 12:31:12,147::task::817::TaskManager.Task::(resourceAcquired) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1207::DEBUG::2012-12-09 12:31:12,148::task::978::TaskManager.Task::(_decref) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::ref 1 aborting False
>Thread-1207::DEBUG::2012-12-09 12:31:12,149::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1207::DEBUG::2012-12-09 12:31:12,150::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1208::DEBUG::2012-12-09 12:31:12,201::task::588::TaskManager.Task::(_updateState) Task=`29040b47-c47a-4c48-9d68-4ea5753755ac`::moving from state init -> state preparing
>Thread-1208::INFO::2012-12-09 12:31:12,202::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1208::INFO::2012-12-09 12:31:12,202::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127401351929', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1208::DEBUG::2012-12-09 12:31:12,202::task::1172::TaskManager.Task::(prepare) Task=`29040b47-c47a-4c48-9d68-4ea5753755ac`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127401351929', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1208::DEBUG::2012-12-09 12:31:12,202::task::588::TaskManager.Task::(_updateState) Task=`29040b47-c47a-4c48-9d68-4ea5753755ac`::moving from state preparing -> state finished
>Thread-1208::DEBUG::2012-12-09 12:31:12,202::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1208::DEBUG::2012-12-09 12:31:12,203::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1208::DEBUG::2012-12-09 12:31:12,203::task::978::TaskManager.Task::(_decref) Task=`29040b47-c47a-4c48-9d68-4ea5753755ac`::ref 0 aborting False
>Thread-1207::DEBUG::2012-12-09 12:31:12,264::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1207::WARNING::2012-12-09 12:31:12,265::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1207::DEBUG::2012-12-09 12:31:12,265::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1207::WARNING::2012-12-09 12:31:12,266::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1207::INFO::2012-12-09 12:31:12,266::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1207::DEBUG::2012-12-09 12:31:12,266::task::1172::TaskManager.Task::(prepare) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1207::DEBUG::2012-12-09 12:31:12,266::task::588::TaskManager.Task::(_updateState) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::moving from state preparing -> state finished
>Thread-1207::DEBUG::2012-12-09 12:31:12,267::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1207::DEBUG::2012-12-09 12:31:12,267::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1207::DEBUG::2012-12-09 12:31:12,267::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1207::DEBUG::2012-12-09 12:31:12,267::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1207::DEBUG::2012-12-09 12:31:12,267::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1207::DEBUG::2012-12-09 12:31:12,268::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1207::DEBUG::2012-12-09 12:31:12,268::task::978::TaskManager.Task::(_decref) Task=`a2917902-0c60-4fe4-a174-6df2dc145887`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:13,723::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:13,774::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272507 s, 37.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:15,781::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:15,828::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022409 s, 45.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:17,834::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:17,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168776 s, 60.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:18,134::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:18,135::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:18,177::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:18,201::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000461091 s, 8.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:18,234::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:18,235::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:18,235::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:18,236::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:31:19,881::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:19,923::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174356 s, 58.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:21,930::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:21,973::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204346 s, 50.1 MB/s\n'; <rc> = 0
>Thread-1214::DEBUG::2012-12-09 12:31:22,478::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1214::DEBUG::2012-12-09 12:31:22,479::task::588::TaskManager.Task::(_updateState) Task=`48d8f1e7-b903-4361-aac2-c27cf47b2749`::moving from state init -> state preparing
>Thread-1214::INFO::2012-12-09 12:31:22,479::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1214::INFO::2012-12-09 12:31:22,480::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1214::DEBUG::2012-12-09 12:31:22,480::task::1172::TaskManager.Task::(prepare) Task=`48d8f1e7-b903-4361-aac2-c27cf47b2749`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1214::DEBUG::2012-12-09 12:31:22,481::task::588::TaskManager.Task::(_updateState) Task=`48d8f1e7-b903-4361-aac2-c27cf47b2749`::moving from state preparing -> state finished
>Thread-1214::DEBUG::2012-12-09 12:31:22,481::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1214::DEBUG::2012-12-09 12:31:22,481::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1214::DEBUG::2012-12-09 12:31:22,482::task::978::TaskManager.Task::(_decref) Task=`48d8f1e7-b903-4361-aac2-c27cf47b2749`::ref 0 aborting False
>Thread-1215::DEBUG::2012-12-09 12:31:22,503::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1215::DEBUG::2012-12-09 12:31:22,504::task::588::TaskManager.Task::(_updateState) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::moving from state init -> state preparing
>Thread-1215::INFO::2012-12-09 12:31:22,504::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1215::DEBUG::2012-12-09 12:31:22,505::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b5c5ce65-1e54-43f4-8672-1f4708bcdc6d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1215::DEBUG::2012-12-09 12:31:22,505::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1215::DEBUG::2012-12-09 12:31:22,506::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1215::DEBUG::2012-12-09 12:31:22,506::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b5c5ce65-1e54-43f4-8672-1f4708bcdc6d`::Granted request
>Thread-1215::DEBUG::2012-12-09 12:31:22,507::task::817::TaskManager.Task::(resourceAcquired) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1215::DEBUG::2012-12-09 12:31:22,507::task::978::TaskManager.Task::(_decref) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::ref 1 aborting False
>Thread-1215::DEBUG::2012-12-09 12:31:22,509::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1215::DEBUG::2012-12-09 12:31:22,510::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1216::DEBUG::2012-12-09 12:31:22,588::task::588::TaskManager.Task::(_updateState) Task=`c4ea5214-5f28-489e-a409-e36d60666df0`::moving from state init -> state preparing
>Thread-1216::INFO::2012-12-09 12:31:22,589::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1216::INFO::2012-12-09 12:31:22,589::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0244829654694', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-1216::DEBUG::2012-12-09 12:31:22,589::task::1172::TaskManager.Task::(prepare) Task=`c4ea5214-5f28-489e-a409-e36d60666df0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0244829654694', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-1216::DEBUG::2012-12-09 12:31:22,590::task::588::TaskManager.Task::(_updateState) Task=`c4ea5214-5f28-489e-a409-e36d60666df0`::moving from state preparing -> state finished
>Thread-1216::DEBUG::2012-12-09 12:31:22,590::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1216::DEBUG::2012-12-09 12:31:22,590::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1216::DEBUG::2012-12-09 12:31:22,591::task::978::TaskManager.Task::(_decref) Task=`c4ea5214-5f28-489e-a409-e36d60666df0`::ref 0 aborting False
>Thread-1215::DEBUG::2012-12-09 12:31:22,613::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1215::WARNING::2012-12-09 12:31:22,614::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1215::DEBUG::2012-12-09 12:31:22,614::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1215::WARNING::2012-12-09 12:31:22,614::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1215::INFO::2012-12-09 12:31:22,615::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1215::DEBUG::2012-12-09 12:31:22,615::task::1172::TaskManager.Task::(prepare) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1215::DEBUG::2012-12-09 12:31:22,615::task::588::TaskManager.Task::(_updateState) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::moving from state preparing -> state finished
>Thread-1215::DEBUG::2012-12-09 12:31:22,616::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1215::DEBUG::2012-12-09 12:31:22,616::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1215::DEBUG::2012-12-09 12:31:22,616::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1215::DEBUG::2012-12-09 12:31:22,616::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1215::DEBUG::2012-12-09 12:31:22,616::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1215::DEBUG::2012-12-09 12:31:22,617::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1215::DEBUG::2012-12-09 12:31:22,617::task::978::TaskManager.Task::(_decref) Task=`0d2bed8c-ee7c-448b-9060-8269dd4c5132`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:23,980::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:24,033::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023042 s, 44.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:26,040::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:26,086::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224663 s, 45.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:28,093::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:28,139::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222217 s, 46.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:31:28,208::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:28,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00107247 s, 3.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:28,237::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:28,238::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:31:28,350::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:28,352::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:28,352::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:28,352::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:31:30,145::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:30,196::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258694 s, 39.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:32,203::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:32,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177311 s, 57.8 MB/s\n'; <rc> = 0
>Thread-1222::DEBUG::2012-12-09 12:31:32,771::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1222::DEBUG::2012-12-09 12:31:32,772::task::588::TaskManager.Task::(_updateState) Task=`54b8044a-3d21-40c9-bef0-7da1ee450088`::moving from state init -> state preparing
>Thread-1222::INFO::2012-12-09 12:31:32,772::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1222::INFO::2012-12-09 12:31:32,772::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1222::DEBUG::2012-12-09 12:31:32,773::task::1172::TaskManager.Task::(prepare) Task=`54b8044a-3d21-40c9-bef0-7da1ee450088`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1222::DEBUG::2012-12-09 12:31:32,773::task::588::TaskManager.Task::(_updateState) Task=`54b8044a-3d21-40c9-bef0-7da1ee450088`::moving from state preparing -> state finished
>Thread-1222::DEBUG::2012-12-09 12:31:32,773::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1222::DEBUG::2012-12-09 12:31:32,774::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1222::DEBUG::2012-12-09 12:31:32,774::task::978::TaskManager.Task::(_decref) Task=`54b8044a-3d21-40c9-bef0-7da1ee450088`::ref 0 aborting False
>Thread-1223::DEBUG::2012-12-09 12:31:32,794::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1223::DEBUG::2012-12-09 12:31:32,795::task::588::TaskManager.Task::(_updateState) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::moving from state init -> state preparing
>Thread-1223::INFO::2012-12-09 12:31:32,795::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1223::DEBUG::2012-12-09 12:31:32,796::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b707f13e-31ba-47d8-b53d-1416bce2cd23`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1223::DEBUG::2012-12-09 12:31:32,796::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1223::DEBUG::2012-12-09 12:31:32,797::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1223::DEBUG::2012-12-09 12:31:32,797::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b707f13e-31ba-47d8-b53d-1416bce2cd23`::Granted request
>Thread-1223::DEBUG::2012-12-09 12:31:32,798::task::817::TaskManager.Task::(resourceAcquired) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1223::DEBUG::2012-12-09 12:31:32,798::task::978::TaskManager.Task::(_decref) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::ref 1 aborting False
>Thread-1223::DEBUG::2012-12-09 12:31:32,800::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1223::DEBUG::2012-12-09 12:31:32,801::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1224::DEBUG::2012-12-09 12:31:32,865::task::588::TaskManager.Task::(_updateState) Task=`1e85ed28-1f1b-4ca6-b624-6243ab50f868`::moving from state init -> state preparing
>Thread-1224::INFO::2012-12-09 12:31:32,865::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1224::INFO::2012-12-09 12:31:32,866::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128831863403', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-1224::DEBUG::2012-12-09 12:31:32,866::task::1172::TaskManager.Task::(prepare) Task=`1e85ed28-1f1b-4ca6-b624-6243ab50f868`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128831863403', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-1224::DEBUG::2012-12-09 12:31:32,866::task::588::TaskManager.Task::(_updateState) Task=`1e85ed28-1f1b-4ca6-b624-6243ab50f868`::moving from state preparing -> state finished
>Thread-1224::DEBUG::2012-12-09 12:31:32,866::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1224::DEBUG::2012-12-09 12:31:32,867::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1224::DEBUG::2012-12-09 12:31:32,867::task::978::TaskManager.Task::(_decref) Task=`1e85ed28-1f1b-4ca6-b624-6243ab50f868`::ref 0 aborting False
>Thread-1223::DEBUG::2012-12-09 12:31:32,911::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1223::WARNING::2012-12-09 12:31:32,912::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1223::DEBUG::2012-12-09 12:31:32,913::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1223::WARNING::2012-12-09 12:31:32,913::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1223::INFO::2012-12-09 12:31:32,913::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1223::DEBUG::2012-12-09 12:31:32,914::task::1172::TaskManager.Task::(prepare) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1223::DEBUG::2012-12-09 12:31:32,914::task::588::TaskManager.Task::(_updateState) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::moving from state preparing -> state finished
>Thread-1223::DEBUG::2012-12-09 12:31:32,914::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1223::DEBUG::2012-12-09 12:31:32,914::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1223::DEBUG::2012-12-09 12:31:32,915::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1223::DEBUG::2012-12-09 12:31:32,915::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1223::DEBUG::2012-12-09 12:31:32,915::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1223::DEBUG::2012-12-09 12:31:32,915::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1223::DEBUG::2012-12-09 12:31:32,915::task::978::TaskManager.Task::(_decref) Task=`c7ef26b5-84d6-4a3a-80ed-25ba32158e57`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:34,250::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:34,299::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023538 s, 43.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:36,306::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:36,348::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190618 s, 53.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:31:38,226::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:38,238::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000763076 s, 5.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:38,353::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:38,355::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:38,355::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:38,404::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159337 s, 64.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:38,469::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:38,471::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:38,472::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:38,472::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:31:40,410::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:40,457::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216881 s, 47.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:42,464::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:42,515::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267628 s, 38.3 MB/s\n'; <rc> = 0
>Thread-1230::DEBUG::2012-12-09 12:31:43,051::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1230::DEBUG::2012-12-09 12:31:43,052::task::588::TaskManager.Task::(_updateState) Task=`0b4c4cc3-4f51-4842-8bf3-ef6ad0c18b19`::moving from state init -> state preparing
>Thread-1230::INFO::2012-12-09 12:31:43,053::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1230::INFO::2012-12-09 12:31:43,053::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1230::DEBUG::2012-12-09 12:31:43,054::task::1172::TaskManager.Task::(prepare) Task=`0b4c4cc3-4f51-4842-8bf3-ef6ad0c18b19`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1230::DEBUG::2012-12-09 12:31:43,054::task::588::TaskManager.Task::(_updateState) Task=`0b4c4cc3-4f51-4842-8bf3-ef6ad0c18b19`::moving from state preparing -> state finished
>Thread-1230::DEBUG::2012-12-09 12:31:43,055::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1230::DEBUG::2012-12-09 12:31:43,055::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1230::DEBUG::2012-12-09 12:31:43,055::task::978::TaskManager.Task::(_decref) Task=`0b4c4cc3-4f51-4842-8bf3-ef6ad0c18b19`::ref 0 aborting False
>Thread-1231::DEBUG::2012-12-09 12:31:43,076::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1231::DEBUG::2012-12-09 12:31:43,077::task::588::TaskManager.Task::(_updateState) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::moving from state init -> state preparing
>Thread-1231::INFO::2012-12-09 12:31:43,077::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1231::DEBUG::2012-12-09 12:31:43,078::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c90b9edd-d05d-4136-966a-4d2d56de37b7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1231::DEBUG::2012-12-09 12:31:43,078::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1231::DEBUG::2012-12-09 12:31:43,079::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1231::DEBUG::2012-12-09 12:31:43,079::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c90b9edd-d05d-4136-966a-4d2d56de37b7`::Granted request
>Thread-1231::DEBUG::2012-12-09 12:31:43,080::task::817::TaskManager.Task::(resourceAcquired) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1231::DEBUG::2012-12-09 12:31:43,080::task::978::TaskManager.Task::(_decref) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::ref 1 aborting False
>Thread-1231::DEBUG::2012-12-09 12:31:43,082::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1231::DEBUG::2012-12-09 12:31:43,082::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1232::DEBUG::2012-12-09 12:31:43,157::task::588::TaskManager.Task::(_updateState) Task=`7c895b35-77f6-4ee3-98d8-9b1cd86d5e0a`::moving from state init -> state preparing
>Thread-1232::INFO::2012-12-09 12:31:43,157::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1232::INFO::2012-12-09 12:31:43,157::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133349895477', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1232::DEBUG::2012-12-09 12:31:43,158::task::1172::TaskManager.Task::(prepare) Task=`7c895b35-77f6-4ee3-98d8-9b1cd86d5e0a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133349895477', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1232::DEBUG::2012-12-09 12:31:43,158::task::588::TaskManager.Task::(_updateState) Task=`7c895b35-77f6-4ee3-98d8-9b1cd86d5e0a`::moving from state preparing -> state finished
>Thread-1232::DEBUG::2012-12-09 12:31:43,158::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1232::DEBUG::2012-12-09 12:31:43,158::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1232::DEBUG::2012-12-09 12:31:43,158::task::978::TaskManager.Task::(_decref) Task=`7c895b35-77f6-4ee3-98d8-9b1cd86d5e0a`::ref 0 aborting False
>Thread-1231::DEBUG::2012-12-09 12:31:43,188::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1231::WARNING::2012-12-09 12:31:43,189::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1231::DEBUG::2012-12-09 12:31:43,189::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1231::WARNING::2012-12-09 12:31:43,189::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1231::INFO::2012-12-09 12:31:43,190::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1231::DEBUG::2012-12-09 12:31:43,190::task::1172::TaskManager.Task::(prepare) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1231::DEBUG::2012-12-09 12:31:43,190::task::588::TaskManager.Task::(_updateState) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::moving from state preparing -> state finished
>Thread-1231::DEBUG::2012-12-09 12:31:43,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1231::DEBUG::2012-12-09 12:31:43,190::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1231::DEBUG::2012-12-09 12:31:43,191::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1231::DEBUG::2012-12-09 12:31:43,191::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1231::DEBUG::2012-12-09 12:31:43,191::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1231::DEBUG::2012-12-09 12:31:43,191::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1231::DEBUG::2012-12-09 12:31:43,191::task::978::TaskManager.Task::(_decref) Task=`cddaa72b-e7de-480b-a33a-7e62ddf9c965`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:44,522::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:44,574::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276875 s, 37.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:46,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:46,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238935 s, 42.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:31:48,243::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:48,258::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000900236 s, 4.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:48,473::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:48,474::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:31:48,591::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:48,592::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:48,592::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:48,593::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:31:48,635::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:48,685::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255932 s, 40.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:50,692::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:50,744::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278301 s, 36.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:52,750::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:52,802::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023668 s, 43.3 MB/s\n'; <rc> = 0
>Thread-1238::DEBUG::2012-12-09 12:31:53,292::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1238::DEBUG::2012-12-09 12:31:53,293::task::588::TaskManager.Task::(_updateState) Task=`1e80f724-c7a0-426c-8644-f27efcd0f57b`::moving from state init -> state preparing
>Thread-1238::INFO::2012-12-09 12:31:53,294::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1238::INFO::2012-12-09 12:31:53,295::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1238::DEBUG::2012-12-09 12:31:53,296::task::1172::TaskManager.Task::(prepare) Task=`1e80f724-c7a0-426c-8644-f27efcd0f57b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1238::DEBUG::2012-12-09 12:31:53,296::task::588::TaskManager.Task::(_updateState) Task=`1e80f724-c7a0-426c-8644-f27efcd0f57b`::moving from state preparing -> state finished
>Thread-1238::DEBUG::2012-12-09 12:31:53,297::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1238::DEBUG::2012-12-09 12:31:53,297::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1238::DEBUG::2012-12-09 12:31:53,297::task::978::TaskManager.Task::(_decref) Task=`1e80f724-c7a0-426c-8644-f27efcd0f57b`::ref 0 aborting False
>Thread-1239::DEBUG::2012-12-09 12:31:53,324::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1239::DEBUG::2012-12-09 12:31:53,324::task::588::TaskManager.Task::(_updateState) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::moving from state init -> state preparing
>Thread-1239::INFO::2012-12-09 12:31:53,325::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1239::DEBUG::2012-12-09 12:31:53,326::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1ba8fcf0-f080-463b-b6d3-39efe226d6d1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1239::DEBUG::2012-12-09 12:31:53,326::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1239::DEBUG::2012-12-09 12:31:53,327::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1239::DEBUG::2012-12-09 12:31:53,327::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1ba8fcf0-f080-463b-b6d3-39efe226d6d1`::Granted request
>Thread-1239::DEBUG::2012-12-09 12:31:53,328::task::817::TaskManager.Task::(resourceAcquired) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1239::DEBUG::2012-12-09 12:31:53,328::task::978::TaskManager.Task::(_decref) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::ref 1 aborting False
>Thread-1239::DEBUG::2012-12-09 12:31:53,330::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1239::DEBUG::2012-12-09 12:31:53,331::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1240::DEBUG::2012-12-09 12:31:53,424::task::588::TaskManager.Task::(_updateState) Task=`ac0401f0-7acf-4ff5-87e2-faf617ea4784`::moving from state init -> state preparing
>Thread-1240::INFO::2012-12-09 12:31:53,424::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1240::INFO::2012-12-09 12:31:53,425::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162568092346', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.8', 'code': 358, 'valid': False}}
>Thread-1240::DEBUG::2012-12-09 12:31:53,425::task::1172::TaskManager.Task::(prepare) Task=`ac0401f0-7acf-4ff5-87e2-faf617ea4784`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162568092346', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.8', 'code': 358, 'valid': False}}
>Thread-1240::DEBUG::2012-12-09 12:31:53,425::task::588::TaskManager.Task::(_updateState) Task=`ac0401f0-7acf-4ff5-87e2-faf617ea4784`::moving from state preparing -> state finished
>Thread-1240::DEBUG::2012-12-09 12:31:53,425::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1240::DEBUG::2012-12-09 12:31:53,426::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1240::DEBUG::2012-12-09 12:31:53,426::task::978::TaskManager.Task::(_decref) Task=`ac0401f0-7acf-4ff5-87e2-faf617ea4784`::ref 0 aborting False
>Thread-1239::DEBUG::2012-12-09 12:31:53,435::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1239::WARNING::2012-12-09 12:31:53,436::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1239::DEBUG::2012-12-09 12:31:53,437::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1239::WARNING::2012-12-09 12:31:53,437::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1239::INFO::2012-12-09 12:31:53,438::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1239::DEBUG::2012-12-09 12:31:53,438::task::1172::TaskManager.Task::(prepare) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1239::DEBUG::2012-12-09 12:31:53,438::task::588::TaskManager.Task::(_updateState) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::moving from state preparing -> state finished
>Thread-1239::DEBUG::2012-12-09 12:31:53,438::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1239::DEBUG::2012-12-09 12:31:53,439::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1239::DEBUG::2012-12-09 12:31:53,439::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1239::DEBUG::2012-12-09 12:31:53,439::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1239::DEBUG::2012-12-09 12:31:53,439::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1239::DEBUG::2012-12-09 12:31:53,439::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1239::DEBUG::2012-12-09 12:31:53,440::task::978::TaskManager.Task::(_decref) Task=`eb1a0461-55e9-40da-86f3-8507b2f11c2e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:31:54,808::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:54,851::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181347 s, 56.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:31:56,858::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:56,906::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0242922 s, 42.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:31:58,263::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:31:58,279::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000904068 s, 4.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:31:58,594::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:31:58,595::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:31:58,712::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:31:58,713::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:31:58,713::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:31:58,714::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:31:58,913::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:31:58,956::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186846 s, 54.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:00,963::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:01,015::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273516 s, 37.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:03,021::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:03,075::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028232 s, 36.3 MB/s\n'; <rc> = 0
>Thread-1246::DEBUG::2012-12-09 12:32:03,564::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1246::DEBUG::2012-12-09 12:32:03,565::task::588::TaskManager.Task::(_updateState) Task=`a559104e-fb9c-47cd-9d7e-6501e9e8b8dc`::moving from state init -> state preparing
>Thread-1246::INFO::2012-12-09 12:32:03,566::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1246::INFO::2012-12-09 12:32:03,567::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1246::DEBUG::2012-12-09 12:32:03,567::task::1172::TaskManager.Task::(prepare) Task=`a559104e-fb9c-47cd-9d7e-6501e9e8b8dc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1246::DEBUG::2012-12-09 12:32:03,568::task::588::TaskManager.Task::(_updateState) Task=`a559104e-fb9c-47cd-9d7e-6501e9e8b8dc`::moving from state preparing -> state finished
>Thread-1246::DEBUG::2012-12-09 12:32:03,568::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1246::DEBUG::2012-12-09 12:32:03,568::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1246::DEBUG::2012-12-09 12:32:03,569::task::978::TaskManager.Task::(_decref) Task=`a559104e-fb9c-47cd-9d7e-6501e9e8b8dc`::ref 0 aborting False
>Thread-1247::DEBUG::2012-12-09 12:32:03,593::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1247::DEBUG::2012-12-09 12:32:03,593::task::588::TaskManager.Task::(_updateState) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::moving from state init -> state preparing
>Thread-1247::INFO::2012-12-09 12:32:03,594::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1247::DEBUG::2012-12-09 12:32:03,595::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b450cd67-4f42-492d-a2a3-7e368b8e56f7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1247::DEBUG::2012-12-09 12:32:03,596::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1247::DEBUG::2012-12-09 12:32:03,596::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1247::DEBUG::2012-12-09 12:32:03,597::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b450cd67-4f42-492d-a2a3-7e368b8e56f7`::Granted request
>Thread-1247::DEBUG::2012-12-09 12:32:03,597::task::817::TaskManager.Task::(resourceAcquired) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1247::DEBUG::2012-12-09 12:32:03,598::task::978::TaskManager.Task::(_decref) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::ref 1 aborting False
>Thread-1247::DEBUG::2012-12-09 12:32:03,599::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1247::DEBUG::2012-12-09 12:32:03,600::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1248::DEBUG::2012-12-09 12:32:03,677::task::588::TaskManager.Task::(_updateState) Task=`46b444dc-6566-4039-9c0d-a190571987f3`::moving from state init -> state preparing
>Thread-1248::INFO::2012-12-09 12:32:03,677::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1248::INFO::2012-12-09 12:32:03,677::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169098377228', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-1248::DEBUG::2012-12-09 12:32:03,678::task::1172::TaskManager.Task::(prepare) Task=`46b444dc-6566-4039-9c0d-a190571987f3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169098377228', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-1248::DEBUG::2012-12-09 12:32:03,678::task::588::TaskManager.Task::(_updateState) Task=`46b444dc-6566-4039-9c0d-a190571987f3`::moving from state preparing -> state finished
>Thread-1248::DEBUG::2012-12-09 12:32:03,678::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1248::DEBUG::2012-12-09 12:32:03,678::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1248::DEBUG::2012-12-09 12:32:03,679::task::978::TaskManager.Task::(_decref) Task=`46b444dc-6566-4039-9c0d-a190571987f3`::ref 0 aborting False
>Thread-1247::DEBUG::2012-12-09 12:32:03,705::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1247::WARNING::2012-12-09 12:32:03,706::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1247::DEBUG::2012-12-09 12:32:03,706::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1247::WARNING::2012-12-09 12:32:03,707::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1247::INFO::2012-12-09 12:32:03,707::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1247::DEBUG::2012-12-09 12:32:03,707::task::1172::TaskManager.Task::(prepare) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1247::DEBUG::2012-12-09 12:32:03,708::task::588::TaskManager.Task::(_updateState) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::moving from state preparing -> state finished
>Thread-1247::DEBUG::2012-12-09 12:32:03,708::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1247::DEBUG::2012-12-09 12:32:03,708::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1247::DEBUG::2012-12-09 12:32:03,708::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1247::DEBUG::2012-12-09 12:32:03,708::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1247::DEBUG::2012-12-09 12:32:03,709::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1247::DEBUG::2012-12-09 12:32:03,709::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1247::DEBUG::2012-12-09 12:32:03,709::task::978::TaskManager.Task::(_decref) Task=`c612e602-c6d1-4b17-9c5a-1e17b386f4f8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:05,082::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:05,127::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218121 s, 46.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:07,134::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:07,179::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203698 s, 50.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:08,284::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:08,299::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00093182 s, 4.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:08,715::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:08,716::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:08,832::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:08,833::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:08,833::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:08,834::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:32:09,186::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:09,230::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193652 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:11,236::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:11,287::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026121 s, 39.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:13,293::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:13,351::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0321062 s, 31.9 MB/s\n'; <rc> = 0
>Thread-1254::DEBUG::2012-12-09 12:32:13,904::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1254::DEBUG::2012-12-09 12:32:13,905::task::588::TaskManager.Task::(_updateState) Task=`0a336def-1203-42ac-9bb5-95c646a4e97a`::moving from state init -> state preparing
>Thread-1254::INFO::2012-12-09 12:32:13,906::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1254::INFO::2012-12-09 12:32:13,906::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1254::DEBUG::2012-12-09 12:32:13,907::task::1172::TaskManager.Task::(prepare) Task=`0a336def-1203-42ac-9bb5-95c646a4e97a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1254::DEBUG::2012-12-09 12:32:13,907::task::588::TaskManager.Task::(_updateState) Task=`0a336def-1203-42ac-9bb5-95c646a4e97a`::moving from state preparing -> state finished
>Thread-1254::DEBUG::2012-12-09 12:32:13,908::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1254::DEBUG::2012-12-09 12:32:13,908::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1254::DEBUG::2012-12-09 12:32:13,908::task::978::TaskManager.Task::(_decref) Task=`0a336def-1203-42ac-9bb5-95c646a4e97a`::ref 0 aborting False
>Thread-1255::DEBUG::2012-12-09 12:32:13,933::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1255::DEBUG::2012-12-09 12:32:13,933::task::588::TaskManager.Task::(_updateState) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::moving from state init -> state preparing
>Thread-1255::INFO::2012-12-09 12:32:13,934::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1255::DEBUG::2012-12-09 12:32:13,935::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aaab5d27-8314-417a-a8a8-c8c278e949d8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1255::DEBUG::2012-12-09 12:32:13,935::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1255::DEBUG::2012-12-09 12:32:13,936::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1255::DEBUG::2012-12-09 12:32:13,936::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aaab5d27-8314-417a-a8a8-c8c278e949d8`::Granted request
>Thread-1255::DEBUG::2012-12-09 12:32:13,937::task::817::TaskManager.Task::(resourceAcquired) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1255::DEBUG::2012-12-09 12:32:13,937::task::978::TaskManager.Task::(_decref) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::ref 1 aborting False
>Thread-1255::DEBUG::2012-12-09 12:32:13,939::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1255::DEBUG::2012-12-09 12:32:13,940::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1256::DEBUG::2012-12-09 12:32:14,001::task::588::TaskManager.Task::(_updateState) Task=`6b1f5e39-f8ec-424d-a884-3fdbcbd5bee1`::moving from state init -> state preparing
>Thread-1256::INFO::2012-12-09 12:32:14,001::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1256::INFO::2012-12-09 12:32:14,001::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167739391327', 'lastCheck': '5.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-1256::DEBUG::2012-12-09 12:32:14,002::task::1172::TaskManager.Task::(prepare) Task=`6b1f5e39-f8ec-424d-a884-3fdbcbd5bee1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167739391327', 'lastCheck': '5.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-1256::DEBUG::2012-12-09 12:32:14,002::task::588::TaskManager.Task::(_updateState) Task=`6b1f5e39-f8ec-424d-a884-3fdbcbd5bee1`::moving from state preparing -> state finished
>Thread-1256::DEBUG::2012-12-09 12:32:14,002::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1256::DEBUG::2012-12-09 12:32:14,002::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1256::DEBUG::2012-12-09 12:32:14,002::task::978::TaskManager.Task::(_decref) Task=`6b1f5e39-f8ec-424d-a884-3fdbcbd5bee1`::ref 0 aborting False
>Thread-1255::DEBUG::2012-12-09 12:32:14,050::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1255::WARNING::2012-12-09 12:32:14,051::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1255::DEBUG::2012-12-09 12:32:14,051::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1255::WARNING::2012-12-09 12:32:14,052::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1255::INFO::2012-12-09 12:32:14,053::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1255::DEBUG::2012-12-09 12:32:14,053::task::1172::TaskManager.Task::(prepare) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1255::DEBUG::2012-12-09 12:32:14,053::task::588::TaskManager.Task::(_updateState) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::moving from state preparing -> state finished
>Thread-1255::DEBUG::2012-12-09 12:32:14,053::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1255::DEBUG::2012-12-09 12:32:14,054::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1255::DEBUG::2012-12-09 12:32:14,054::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1255::DEBUG::2012-12-09 12:32:14,054::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1255::DEBUG::2012-12-09 12:32:14,055::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1255::DEBUG::2012-12-09 12:32:14,055::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1255::DEBUG::2012-12-09 12:32:14,055::task::978::TaskManager.Task::(_decref) Task=`db2cb467-0b59-429c-a9ad-8e041d59b3d6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:15,358::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:15,404::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159354 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:17,411::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:17,458::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224206 s, 45.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:18,304::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:18,318::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000803661 s, 5.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:18,835::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:18,836::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:18,951::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:18,952::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:18,952::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:18,953::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:32:19,465::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:19,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261764 s, 39.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:21,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:21,569::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216215 s, 47.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:23,575::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:23,626::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261316 s, 39.2 MB/s\n'; <rc> = 0
>Thread-1262::DEBUG::2012-12-09 12:32:24,145::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1262::DEBUG::2012-12-09 12:32:24,146::task::588::TaskManager.Task::(_updateState) Task=`46fd62af-0150-44ef-8d79-9d1804f2e533`::moving from state init -> state preparing
>Thread-1262::INFO::2012-12-09 12:32:24,147::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1262::INFO::2012-12-09 12:32:24,147::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1262::DEBUG::2012-12-09 12:32:24,148::task::1172::TaskManager.Task::(prepare) Task=`46fd62af-0150-44ef-8d79-9d1804f2e533`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1262::DEBUG::2012-12-09 12:32:24,148::task::588::TaskManager.Task::(_updateState) Task=`46fd62af-0150-44ef-8d79-9d1804f2e533`::moving from state preparing -> state finished
>Thread-1262::DEBUG::2012-12-09 12:32:24,149::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1262::DEBUG::2012-12-09 12:32:24,149::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1262::DEBUG::2012-12-09 12:32:24,150::task::978::TaskManager.Task::(_decref) Task=`46fd62af-0150-44ef-8d79-9d1804f2e533`::ref 0 aborting False
>Thread-1263::DEBUG::2012-12-09 12:32:24,173::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1263::DEBUG::2012-12-09 12:32:24,174::task::588::TaskManager.Task::(_updateState) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::moving from state init -> state preparing
>Thread-1263::INFO::2012-12-09 12:32:24,175::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1263::DEBUG::2012-12-09 12:32:24,175::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`77b5c041-6068-40e8-aacc-f55a6d48438e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1263::DEBUG::2012-12-09 12:32:24,176::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1263::DEBUG::2012-12-09 12:32:24,176::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1263::DEBUG::2012-12-09 12:32:24,177::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`77b5c041-6068-40e8-aacc-f55a6d48438e`::Granted request
>Thread-1263::DEBUG::2012-12-09 12:32:24,177::task::817::TaskManager.Task::(resourceAcquired) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1263::DEBUG::2012-12-09 12:32:24,178::task::978::TaskManager.Task::(_decref) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::ref 1 aborting False
>Thread-1263::DEBUG::2012-12-09 12:32:24,179::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1263::DEBUG::2012-12-09 12:32:24,180::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1264::DEBUG::2012-12-09 12:32:24,236::task::588::TaskManager.Task::(_updateState) Task=`903b1324-e00b-410f-ba61-14ef0b8ae972`::moving from state init -> state preparing
>Thread-1264::INFO::2012-12-09 12:32:24,236::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1264::INFO::2012-12-09 12:32:24,237::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154230594635', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-1264::DEBUG::2012-12-09 12:32:24,237::task::1172::TaskManager.Task::(prepare) Task=`903b1324-e00b-410f-ba61-14ef0b8ae972`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154230594635', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-1264::DEBUG::2012-12-09 12:32:24,237::task::588::TaskManager.Task::(_updateState) Task=`903b1324-e00b-410f-ba61-14ef0b8ae972`::moving from state preparing -> state finished
>Thread-1264::DEBUG::2012-12-09 12:32:24,238::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1264::DEBUG::2012-12-09 12:32:24,238::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1264::DEBUG::2012-12-09 12:32:24,238::task::978::TaskManager.Task::(_decref) Task=`903b1324-e00b-410f-ba61-14ef0b8ae972`::ref 0 aborting False
>Thread-1263::DEBUG::2012-12-09 12:32:24,293::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1263::WARNING::2012-12-09 12:32:24,295::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1263::DEBUG::2012-12-09 12:32:24,296::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1263::WARNING::2012-12-09 12:32:24,297::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1263::INFO::2012-12-09 12:32:24,298::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1263::DEBUG::2012-12-09 12:32:24,298::task::1172::TaskManager.Task::(prepare) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1263::DEBUG::2012-12-09 12:32:24,299::task::588::TaskManager.Task::(_updateState) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::moving from state preparing -> state finished
>Thread-1263::DEBUG::2012-12-09 12:32:24,299::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1263::DEBUG::2012-12-09 12:32:24,299::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1263::DEBUG::2012-12-09 12:32:24,300::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1263::DEBUG::2012-12-09 12:32:24,300::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1263::DEBUG::2012-12-09 12:32:24,301::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1263::DEBUG::2012-12-09 12:32:24,301::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1263::DEBUG::2012-12-09 12:32:24,302::task::978::TaskManager.Task::(_decref) Task=`076f25eb-9449-4e51-a5eb-0d058718852f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:25,632::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:25,684::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270982 s, 37.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:27,691::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:27,744::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0286149 s, 35.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:28,324::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:28,339::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000941791 s, 4.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:28,954::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:28,955::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:29,066::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:29,067::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:29,068::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:29,068::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:32:29,750::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:29,796::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223987 s, 45.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:31,803::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:31,848::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215145 s, 47.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:33,855::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:33,898::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185179 s, 55.3 MB/s\n'; <rc> = 0
>Thread-1270::DEBUG::2012-12-09 12:32:34,412::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1270::DEBUG::2012-12-09 12:32:34,413::task::588::TaskManager.Task::(_updateState) Task=`4dca22ce-a733-4a2d-940b-2fd5c87a19cb`::moving from state init -> state preparing
>Thread-1270::INFO::2012-12-09 12:32:34,413::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1270::INFO::2012-12-09 12:32:34,414::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1270::DEBUG::2012-12-09 12:32:34,415::task::1172::TaskManager.Task::(prepare) Task=`4dca22ce-a733-4a2d-940b-2fd5c87a19cb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1270::DEBUG::2012-12-09 12:32:34,415::task::588::TaskManager.Task::(_updateState) Task=`4dca22ce-a733-4a2d-940b-2fd5c87a19cb`::moving from state preparing -> state finished
>Thread-1270::DEBUG::2012-12-09 12:32:34,415::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1270::DEBUG::2012-12-09 12:32:34,416::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1270::DEBUG::2012-12-09 12:32:34,416::task::978::TaskManager.Task::(_decref) Task=`4dca22ce-a733-4a2d-940b-2fd5c87a19cb`::ref 0 aborting False
>Thread-1271::DEBUG::2012-12-09 12:32:34,441::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1271::DEBUG::2012-12-09 12:32:34,442::task::588::TaskManager.Task::(_updateState) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::moving from state init -> state preparing
>Thread-1271::INFO::2012-12-09 12:32:34,442::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1271::DEBUG::2012-12-09 12:32:34,443::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aa2483f4-19e3-455f-bb28-4c70b723793c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1271::DEBUG::2012-12-09 12:32:34,443::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1271::DEBUG::2012-12-09 12:32:34,444::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1271::DEBUG::2012-12-09 12:32:34,444::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aa2483f4-19e3-455f-bb28-4c70b723793c`::Granted request
>Thread-1271::DEBUG::2012-12-09 12:32:34,445::task::817::TaskManager.Task::(resourceAcquired) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1271::DEBUG::2012-12-09 12:32:34,445::task::978::TaskManager.Task::(_decref) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::ref 1 aborting False
>Thread-1271::DEBUG::2012-12-09 12:32:34,447::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1271::DEBUG::2012-12-09 12:32:34,448::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1272::DEBUG::2012-12-09 12:32:34,511::task::588::TaskManager.Task::(_updateState) Task=`4e566557-0770-40dd-9a3f-c8af198933a3`::moving from state init -> state preparing
>Thread-1272::INFO::2012-12-09 12:32:34,511::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1272::INFO::2012-12-09 12:32:34,512::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163629055023', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1272::DEBUG::2012-12-09 12:32:34,512::task::1172::TaskManager.Task::(prepare) Task=`4e566557-0770-40dd-9a3f-c8af198933a3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163629055023', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1272::DEBUG::2012-12-09 12:32:34,512::task::588::TaskManager.Task::(_updateState) Task=`4e566557-0770-40dd-9a3f-c8af198933a3`::moving from state preparing -> state finished
>Thread-1272::DEBUG::2012-12-09 12:32:34,512::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1272::DEBUG::2012-12-09 12:32:34,513::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1272::DEBUG::2012-12-09 12:32:34,513::task::978::TaskManager.Task::(_decref) Task=`4e566557-0770-40dd-9a3f-c8af198933a3`::ref 0 aborting False
>Thread-1271::DEBUG::2012-12-09 12:32:34,566::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1271::WARNING::2012-12-09 12:32:34,568::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1271::DEBUG::2012-12-09 12:32:34,568::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1271::WARNING::2012-12-09 12:32:34,569::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1271::INFO::2012-12-09 12:32:34,570::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1271::DEBUG::2012-12-09 12:32:34,571::task::1172::TaskManager.Task::(prepare) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1271::DEBUG::2012-12-09 12:32:34,571::task::588::TaskManager.Task::(_updateState) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::moving from state preparing -> state finished
>Thread-1271::DEBUG::2012-12-09 12:32:34,571::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1271::DEBUG::2012-12-09 12:32:34,572::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1271::DEBUG::2012-12-09 12:32:34,572::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1271::DEBUG::2012-12-09 12:32:34,573::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1271::DEBUG::2012-12-09 12:32:34,573::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1271::DEBUG::2012-12-09 12:32:34,574::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1271::DEBUG::2012-12-09 12:32:34,574::task::978::TaskManager.Task::(_decref) Task=`53160166-fadf-41bf-81d1-0c823a807e95`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:35,904::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:35,955::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260086 s, 39.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:37,961::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:38,009::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0232265 s, 44.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:38,344::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:38,355::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000453492 s, 9.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:39,070::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:39,071::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:39,187::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:39,189::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:39,190::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:39,191::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:32:40,016::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:40,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256911 s, 39.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:42,073::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:42,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216093 s, 47.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:44,126::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:44,165::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175607 s, 58.3 MB/s\n'; <rc> = 0
>Thread-1278::DEBUG::2012-12-09 12:32:44,707::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1278::DEBUG::2012-12-09 12:32:44,707::task::588::TaskManager.Task::(_updateState) Task=`5967b538-5359-4faf-b16f-e5ca9c063522`::moving from state init -> state preparing
>Thread-1278::INFO::2012-12-09 12:32:44,708::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1278::INFO::2012-12-09 12:32:44,708::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1278::DEBUG::2012-12-09 12:32:44,708::task::1172::TaskManager.Task::(prepare) Task=`5967b538-5359-4faf-b16f-e5ca9c063522`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1278::DEBUG::2012-12-09 12:32:44,708::task::588::TaskManager.Task::(_updateState) Task=`5967b538-5359-4faf-b16f-e5ca9c063522`::moving from state preparing -> state finished
>Thread-1278::DEBUG::2012-12-09 12:32:44,709::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1278::DEBUG::2012-12-09 12:32:44,709::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1278::DEBUG::2012-12-09 12:32:44,709::task::978::TaskManager.Task::(_decref) Task=`5967b538-5359-4faf-b16f-e5ca9c063522`::ref 0 aborting False
>Thread-1279::DEBUG::2012-12-09 12:32:44,722::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1279::DEBUG::2012-12-09 12:32:44,722::task::588::TaskManager.Task::(_updateState) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::moving from state init -> state preparing
>Thread-1279::INFO::2012-12-09 12:32:44,722::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1279::DEBUG::2012-12-09 12:32:44,723::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3241f298-bab8-4245-ba1a-5b567a6c94d3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1279::DEBUG::2012-12-09 12:32:44,723::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1279::DEBUG::2012-12-09 12:32:44,723::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1279::DEBUG::2012-12-09 12:32:44,724::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3241f298-bab8-4245-ba1a-5b567a6c94d3`::Granted request
>Thread-1279::DEBUG::2012-12-09 12:32:44,724::task::817::TaskManager.Task::(resourceAcquired) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1279::DEBUG::2012-12-09 12:32:44,724::task::978::TaskManager.Task::(_decref) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::ref 1 aborting False
>Thread-1279::DEBUG::2012-12-09 12:32:44,725::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1279::DEBUG::2012-12-09 12:32:44,725::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1279::DEBUG::2012-12-09 12:32:44,829::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1279::WARNING::2012-12-09 12:32:44,831::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1279::DEBUG::2012-12-09 12:32:44,832::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1279::WARNING::2012-12-09 12:32:44,832::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1279::INFO::2012-12-09 12:32:44,832::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1279::DEBUG::2012-12-09 12:32:44,833::task::1172::TaskManager.Task::(prepare) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1279::DEBUG::2012-12-09 12:32:44,833::task::588::TaskManager.Task::(_updateState) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::moving from state preparing -> state finished
>Thread-1279::DEBUG::2012-12-09 12:32:44,833::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1279::DEBUG::2012-12-09 12:32:44,833::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1279::DEBUG::2012-12-09 12:32:44,834::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1279::DEBUG::2012-12-09 12:32:44,834::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1279::DEBUG::2012-12-09 12:32:44,834::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1279::DEBUG::2012-12-09 12:32:44,834::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1279::DEBUG::2012-12-09 12:32:44,834::task::978::TaskManager.Task::(_decref) Task=`610c4588-5f83-4c2d-a7d6-e99f06e464c5`::ref 0 aborting False
>Thread-1280::DEBUG::2012-12-09 12:32:44,852::task::588::TaskManager.Task::(_updateState) Task=`250fde2e-e913-4077-9a17-f5f7f153b462`::moving from state init -> state preparing
>Thread-1280::INFO::2012-12-09 12:32:44,853::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1280::INFO::2012-12-09 12:32:44,853::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011647939682', 'lastCheck': '6.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-1280::DEBUG::2012-12-09 12:32:44,853::task::1172::TaskManager.Task::(prepare) Task=`250fde2e-e913-4077-9a17-f5f7f153b462`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011647939682', 'lastCheck': '6.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-1280::DEBUG::2012-12-09 12:32:44,853::task::588::TaskManager.Task::(_updateState) Task=`250fde2e-e913-4077-9a17-f5f7f153b462`::moving from state preparing -> state finished
>Thread-1280::DEBUG::2012-12-09 12:32:44,853::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1280::DEBUG::2012-12-09 12:32:44,854::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1280::DEBUG::2012-12-09 12:32:44,854::task::978::TaskManager.Task::(_decref) Task=`250fde2e-e913-4077-9a17-f5f7f153b462`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:46,172::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:46,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230131 s, 44.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:48,227::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:48,271::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019933 s, 51.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:48,358::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:48,371::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000543956 s, 7.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:49,192::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:49,193::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:49,310::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:49,312::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:49,312::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:49,312::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:32:50,278::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:50,326::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024382 s, 42.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:52,332::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:52,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276158 s, 37.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:54,393::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:54,439::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214707 s, 47.7 MB/s\n'; <rc> = 0
>Thread-1286::DEBUG::2012-12-09 12:32:55,057::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1286::DEBUG::2012-12-09 12:32:55,057::task::588::TaskManager.Task::(_updateState) Task=`1c9bb035-8891-445d-81a6-3b66408821e8`::moving from state init -> state preparing
>Thread-1286::INFO::2012-12-09 12:32:55,057::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1286::INFO::2012-12-09 12:32:55,058::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1286::DEBUG::2012-12-09 12:32:55,058::task::1172::TaskManager.Task::(prepare) Task=`1c9bb035-8891-445d-81a6-3b66408821e8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1286::DEBUG::2012-12-09 12:32:55,058::task::588::TaskManager.Task::(_updateState) Task=`1c9bb035-8891-445d-81a6-3b66408821e8`::moving from state preparing -> state finished
>Thread-1286::DEBUG::2012-12-09 12:32:55,058::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1286::DEBUG::2012-12-09 12:32:55,058::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1286::DEBUG::2012-12-09 12:32:55,058::task::978::TaskManager.Task::(_decref) Task=`1c9bb035-8891-445d-81a6-3b66408821e8`::ref 0 aborting False
>Thread-1287::DEBUG::2012-12-09 12:32:55,082::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1287::DEBUG::2012-12-09 12:32:55,083::task::588::TaskManager.Task::(_updateState) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::moving from state init -> state preparing
>Thread-1287::INFO::2012-12-09 12:32:55,083::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1287::DEBUG::2012-12-09 12:32:55,084::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`78b3fab7-7e3f-46aa-8411-41bca54cfa46`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1287::DEBUG::2012-12-09 12:32:55,084::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1287::DEBUG::2012-12-09 12:32:55,085::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1287::DEBUG::2012-12-09 12:32:55,085::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`78b3fab7-7e3f-46aa-8411-41bca54cfa46`::Granted request
>Thread-1287::DEBUG::2012-12-09 12:32:55,086::task::817::TaskManager.Task::(resourceAcquired) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1287::DEBUG::2012-12-09 12:32:55,086::task::978::TaskManager.Task::(_decref) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::ref 1 aborting False
>Thread-1287::DEBUG::2012-12-09 12:32:55,088::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1287::DEBUG::2012-12-09 12:32:55,089::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1287::DEBUG::2012-12-09 12:32:55,198::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1287::WARNING::2012-12-09 12:32:55,201::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1287::DEBUG::2012-12-09 12:32:55,201::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1287::WARNING::2012-12-09 12:32:55,202::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1287::INFO::2012-12-09 12:32:55,202::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1287::DEBUG::2012-12-09 12:32:55,203::task::1172::TaskManager.Task::(prepare) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1287::DEBUG::2012-12-09 12:32:55,203::task::588::TaskManager.Task::(_updateState) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::moving from state preparing -> state finished
>Thread-1287::DEBUG::2012-12-09 12:32:55,203::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1287::DEBUG::2012-12-09 12:32:55,204::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1287::DEBUG::2012-12-09 12:32:55,204::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1287::DEBUG::2012-12-09 12:32:55,204::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1287::DEBUG::2012-12-09 12:32:55,205::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1287::DEBUG::2012-12-09 12:32:55,205::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1287::DEBUG::2012-12-09 12:32:55,205::task::978::TaskManager.Task::(_decref) Task=`4486838f-369c-455a-af5e-d729c6bfa083`::ref 0 aborting False
>Thread-1288::DEBUG::2012-12-09 12:32:55,283::task::588::TaskManager.Task::(_updateState) Task=`8982ece4-f12a-47d9-a220-526db68a5a7e`::moving from state init -> state preparing
>Thread-1288::INFO::2012-12-09 12:32:55,283::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1288::INFO::2012-12-09 12:32:55,283::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0139000415802', 'lastCheck': '6.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-1288::DEBUG::2012-12-09 12:32:55,284::task::1172::TaskManager.Task::(prepare) Task=`8982ece4-f12a-47d9-a220-526db68a5a7e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0139000415802', 'lastCheck': '6.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-1288::DEBUG::2012-12-09 12:32:55,284::task::588::TaskManager.Task::(_updateState) Task=`8982ece4-f12a-47d9-a220-526db68a5a7e`::moving from state preparing -> state finished
>Thread-1288::DEBUG::2012-12-09 12:32:55,284::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1288::DEBUG::2012-12-09 12:32:55,284::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1288::DEBUG::2012-12-09 12:32:55,284::task::978::TaskManager.Task::(_decref) Task=`8982ece4-f12a-47d9-a220-526db68a5a7e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:32:56,446::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:56,496::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258106 s, 39.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:32:58,376::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:32:58,384::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000357918 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:32:58,502::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:32:58,539::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015282 s, 67.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:32:59,313::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:32:59,314::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:32:59,412::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:32:59,413::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:32:59,413::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:32:59,414::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:00,546::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:00,585::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158866 s, 64.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:02,591::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:02,630::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169168 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:04,637::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:04,703::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027482 s, 37.3 MB/s\n'; <rc> = 0
>Thread-1294::DEBUG::2012-12-09 12:33:05,285::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1294::DEBUG::2012-12-09 12:33:05,285::task::588::TaskManager.Task::(_updateState) Task=`dc187191-98ab-4d42-922a-5b04e05d108c`::moving from state init -> state preparing
>Thread-1294::INFO::2012-12-09 12:33:05,286::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1294::INFO::2012-12-09 12:33:05,286::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1294::DEBUG::2012-12-09 12:33:05,286::task::1172::TaskManager.Task::(prepare) Task=`dc187191-98ab-4d42-922a-5b04e05d108c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1294::DEBUG::2012-12-09 12:33:05,286::task::588::TaskManager.Task::(_updateState) Task=`dc187191-98ab-4d42-922a-5b04e05d108c`::moving from state preparing -> state finished
>Thread-1294::DEBUG::2012-12-09 12:33:05,286::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1294::DEBUG::2012-12-09 12:33:05,287::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1294::DEBUG::2012-12-09 12:33:05,287::task::978::TaskManager.Task::(_decref) Task=`dc187191-98ab-4d42-922a-5b04e05d108c`::ref 0 aborting False
>Thread-1295::DEBUG::2012-12-09 12:33:05,299::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1295::DEBUG::2012-12-09 12:33:05,299::task::588::TaskManager.Task::(_updateState) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::moving from state init -> state preparing
>Thread-1295::INFO::2012-12-09 12:33:05,300::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1295::DEBUG::2012-12-09 12:33:05,300::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c66a1e7d-9fa7-42b7-b03b-664fdbb6dd6c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1295::DEBUG::2012-12-09 12:33:05,300::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1295::DEBUG::2012-12-09 12:33:05,300::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1295::DEBUG::2012-12-09 12:33:05,301::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c66a1e7d-9fa7-42b7-b03b-664fdbb6dd6c`::Granted request
>Thread-1295::DEBUG::2012-12-09 12:33:05,301::task::817::TaskManager.Task::(resourceAcquired) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1295::DEBUG::2012-12-09 12:33:05,301::task::978::TaskManager.Task::(_decref) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::ref 1 aborting False
>Thread-1295::DEBUG::2012-12-09 12:33:05,302::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1295::DEBUG::2012-12-09 12:33:05,302::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1295::DEBUG::2012-12-09 12:33:05,414::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1295::WARNING::2012-12-09 12:33:05,415::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1295::DEBUG::2012-12-09 12:33:05,416::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1295::WARNING::2012-12-09 12:33:05,416::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1295::INFO::2012-12-09 12:33:05,417::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1295::DEBUG::2012-12-09 12:33:05,417::task::1172::TaskManager.Task::(prepare) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1295::DEBUG::2012-12-09 12:33:05,417::task::588::TaskManager.Task::(_updateState) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::moving from state preparing -> state finished
>Thread-1295::DEBUG::2012-12-09 12:33:05,417::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1295::DEBUG::2012-12-09 12:33:05,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1295::DEBUG::2012-12-09 12:33:05,418::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1295::DEBUG::2012-12-09 12:33:05,418::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1295::DEBUG::2012-12-09 12:33:05,418::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1295::DEBUG::2012-12-09 12:33:05,419::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1295::DEBUG::2012-12-09 12:33:05,419::task::978::TaskManager.Task::(_decref) Task=`97d9299a-a1be-4b42-9fa0-04add84f2a66`::ref 0 aborting False
>Thread-1296::DEBUG::2012-12-09 12:33:05,498::task::588::TaskManager.Task::(_updateState) Task=`e978e53b-9582-452a-ad6e-773cd1a4d981`::moving from state init -> state preparing
>Thread-1296::INFO::2012-12-09 12:33:05,498::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1296::INFO::2012-12-09 12:33:05,499::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00786113739014', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-1296::DEBUG::2012-12-09 12:33:05,499::task::1172::TaskManager.Task::(prepare) Task=`e978e53b-9582-452a-ad6e-773cd1a4d981`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00786113739014', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-1296::DEBUG::2012-12-09 12:33:05,499::task::588::TaskManager.Task::(_updateState) Task=`e978e53b-9582-452a-ad6e-773cd1a4d981`::moving from state preparing -> state finished
>Thread-1296::DEBUG::2012-12-09 12:33:05,499::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1296::DEBUG::2012-12-09 12:33:05,499::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1296::DEBUG::2012-12-09 12:33:05,499::task::978::TaskManager.Task::(_decref) Task=`e978e53b-9582-452a-ad6e-773cd1a4d981`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:06,712::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:06,750::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163189 s, 62.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:08,387::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:08,396::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000417016 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:08,757::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:08,807::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254354 s, 40.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:09,416::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:09,417::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:33:09,531::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:33:09,532::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:33:09,533::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:33:09,533::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:10,814::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:10,864::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262124 s, 39.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:12,869::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:12,921::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243974 s, 42.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:14,928::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:14,970::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183842 s, 55.7 MB/s\n'; <rc> = 0
>Thread-1302::DEBUG::2012-12-09 12:33:16,686::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1302::DEBUG::2012-12-09 12:33:16,687::task::588::TaskManager.Task::(_updateState) Task=`05aa16f1-68df-48bc-9b2f-fe7501ddd4b7`::moving from state init -> state preparing
>Thread-1302::INFO::2012-12-09 12:33:16,687::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1302::INFO::2012-12-09 12:33:16,688::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1302::DEBUG::2012-12-09 12:33:16,688::task::1172::TaskManager.Task::(prepare) Task=`05aa16f1-68df-48bc-9b2f-fe7501ddd4b7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1302::DEBUG::2012-12-09 12:33:16,689::task::588::TaskManager.Task::(_updateState) Task=`05aa16f1-68df-48bc-9b2f-fe7501ddd4b7`::moving from state preparing -> state finished
>Thread-1302::DEBUG::2012-12-09 12:33:16,689::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1302::DEBUG::2012-12-09 12:33:16,690::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1302::DEBUG::2012-12-09 12:33:16,690::task::978::TaskManager.Task::(_decref) Task=`05aa16f1-68df-48bc-9b2f-fe7501ddd4b7`::ref 0 aborting False
>Thread-1303::DEBUG::2012-12-09 12:33:16,708::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1303::DEBUG::2012-12-09 12:33:16,709::task::588::TaskManager.Task::(_updateState) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::moving from state init -> state preparing
>Thread-1303::INFO::2012-12-09 12:33:16,710::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1303::DEBUG::2012-12-09 12:33:16,710::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`adc7e6cd-c1c1-4315-93cb-ca1dbc2391fb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1303::DEBUG::2012-12-09 12:33:16,711::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1303::DEBUG::2012-12-09 12:33:16,711::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1303::DEBUG::2012-12-09 12:33:16,712::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`adc7e6cd-c1c1-4315-93cb-ca1dbc2391fb`::Granted request
>Thread-1303::DEBUG::2012-12-09 12:33:16,712::task::817::TaskManager.Task::(resourceAcquired) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1303::DEBUG::2012-12-09 12:33:16,712::task::978::TaskManager.Task::(_decref) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::ref 1 aborting False
>Thread-1303::DEBUG::2012-12-09 12:33:16,713::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1303::DEBUG::2012-12-09 12:33:16,713::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1304::DEBUG::2012-12-09 12:33:16,794::task::588::TaskManager.Task::(_updateState) Task=`3bc8da98-ab1c-4dfd-b81b-9d0b9da101b3`::moving from state init -> state preparing
>Thread-1304::INFO::2012-12-09 12:33:16,794::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1304::INFO::2012-12-09 12:33:16,795::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00983309745789', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1304::DEBUG::2012-12-09 12:33:16,795::task::1172::TaskManager.Task::(prepare) Task=`3bc8da98-ab1c-4dfd-b81b-9d0b9da101b3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00983309745789', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1304::DEBUG::2012-12-09 12:33:16,795::task::588::TaskManager.Task::(_updateState) Task=`3bc8da98-ab1c-4dfd-b81b-9d0b9da101b3`::moving from state preparing -> state finished
>Thread-1304::DEBUG::2012-12-09 12:33:16,796::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1304::DEBUG::2012-12-09 12:33:16,796::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1304::DEBUG::2012-12-09 12:33:16,796::task::978::TaskManager.Task::(_decref) Task=`3bc8da98-ab1c-4dfd-b81b-9d0b9da101b3`::ref 0 aborting False
>Thread-1303::DEBUG::2012-12-09 12:33:16,817::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1303::WARNING::2012-12-09 12:33:16,818::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1303::DEBUG::2012-12-09 12:33:16,818::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1303::WARNING::2012-12-09 12:33:16,819::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1303::INFO::2012-12-09 12:33:16,819::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1303::DEBUG::2012-12-09 12:33:16,819::task::1172::TaskManager.Task::(prepare) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1303::DEBUG::2012-12-09 12:33:16,819::task::588::TaskManager.Task::(_updateState) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::moving from state preparing -> state finished
>Thread-1303::DEBUG::2012-12-09 12:33:16,820::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1303::DEBUG::2012-12-09 12:33:16,820::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1303::DEBUG::2012-12-09 12:33:16,820::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1303::DEBUG::2012-12-09 12:33:16,820::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1303::DEBUG::2012-12-09 12:33:16,821::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1303::DEBUG::2012-12-09 12:33:16,821::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1303::DEBUG::2012-12-09 12:33:16,821::task::978::TaskManager.Task::(_decref) Task=`db2b947a-278b-4b66-aeef-d837b8a48fee`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:16,977::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:17,020::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185216 s, 55.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:18,407::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:18,422::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000727491 s, 5.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:19,026::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:19,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256444 s, 39.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:19,534::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:19,535::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:33:19,652::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:33:19,653::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:33:19,653::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:33:19,654::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:21,084::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:21,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270385 s, 37.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:23,143::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:23,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212467 s, 48.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:25,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:25,246::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254406 s, 40.3 MB/s\n'; <rc> = 0
>Thread-1310::DEBUG::2012-12-09 12:33:26,914::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1310::DEBUG::2012-12-09 12:33:26,915::task::588::TaskManager.Task::(_updateState) Task=`a64eaa53-9bdb-41e5-8d3a-abc4b0eb80ca`::moving from state init -> state preparing
>Thread-1310::INFO::2012-12-09 12:33:26,915::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1310::INFO::2012-12-09 12:33:26,916::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1310::DEBUG::2012-12-09 12:33:26,916::task::1172::TaskManager.Task::(prepare) Task=`a64eaa53-9bdb-41e5-8d3a-abc4b0eb80ca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1310::DEBUG::2012-12-09 12:33:26,917::task::588::TaskManager.Task::(_updateState) Task=`a64eaa53-9bdb-41e5-8d3a-abc4b0eb80ca`::moving from state preparing -> state finished
>Thread-1310::DEBUG::2012-12-09 12:33:26,917::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1310::DEBUG::2012-12-09 12:33:26,918::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1310::DEBUG::2012-12-09 12:33:26,918::task::978::TaskManager.Task::(_decref) Task=`a64eaa53-9bdb-41e5-8d3a-abc4b0eb80ca`::ref 0 aborting False
>Thread-1311::DEBUG::2012-12-09 12:33:26,941::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1311::DEBUG::2012-12-09 12:33:26,941::task::588::TaskManager.Task::(_updateState) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::moving from state init -> state preparing
>Thread-1311::INFO::2012-12-09 12:33:26,942::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1311::DEBUG::2012-12-09 12:33:26,942::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8504db6b-1fe2-4b3a-9856-5d6cd53b568b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1311::DEBUG::2012-12-09 12:33:26,943::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1311::DEBUG::2012-12-09 12:33:26,943::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1311::DEBUG::2012-12-09 12:33:26,944::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8504db6b-1fe2-4b3a-9856-5d6cd53b568b`::Granted request
>Thread-1311::DEBUG::2012-12-09 12:33:26,944::task::817::TaskManager.Task::(resourceAcquired) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1311::DEBUG::2012-12-09 12:33:26,945::task::978::TaskManager.Task::(_decref) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::ref 1 aborting False
>Thread-1311::DEBUG::2012-12-09 12:33:26,946::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1311::DEBUG::2012-12-09 12:33:26,947::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1312::DEBUG::2012-12-09 12:33:27,003::task::588::TaskManager.Task::(_updateState) Task=`79ffbfc8-c6f5-4f91-8f1f-24fac1b0326f`::moving from state init -> state preparing
>Thread-1312::INFO::2012-12-09 12:33:27,003::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1312::INFO::2012-12-09 12:33:27,004::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158851146698', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1312::DEBUG::2012-12-09 12:33:27,004::task::1172::TaskManager.Task::(prepare) Task=`79ffbfc8-c6f5-4f91-8f1f-24fac1b0326f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158851146698', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1312::DEBUG::2012-12-09 12:33:27,004::task::588::TaskManager.Task::(_updateState) Task=`79ffbfc8-c6f5-4f91-8f1f-24fac1b0326f`::moving from state preparing -> state finished
>Thread-1312::DEBUG::2012-12-09 12:33:27,004::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1312::DEBUG::2012-12-09 12:33:27,005::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1312::DEBUG::2012-12-09 12:33:27,005::task::978::TaskManager.Task::(_decref) Task=`79ffbfc8-c6f5-4f91-8f1f-24fac1b0326f`::ref 0 aborting False
>Thread-1311::DEBUG::2012-12-09 12:33:27,065::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1311::WARNING::2012-12-09 12:33:27,066::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1311::DEBUG::2012-12-09 12:33:27,066::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1311::WARNING::2012-12-09 12:33:27,067::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1311::INFO::2012-12-09 12:33:27,067::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1311::DEBUG::2012-12-09 12:33:27,067::task::1172::TaskManager.Task::(prepare) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1311::DEBUG::2012-12-09 12:33:27,067::task::588::TaskManager.Task::(_updateState) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::moving from state preparing -> state finished
>Thread-1311::DEBUG::2012-12-09 12:33:27,067::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1311::DEBUG::2012-12-09 12:33:27,068::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1311::DEBUG::2012-12-09 12:33:27,068::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1311::DEBUG::2012-12-09 12:33:27,068::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1311::DEBUG::2012-12-09 12:33:27,068::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1311::DEBUG::2012-12-09 12:33:27,069::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1311::DEBUG::2012-12-09 12:33:27,069::task::978::TaskManager.Task::(_decref) Task=`7c1c3b86-366f-467f-9578-54429f1085de`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:27,253::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:27,305::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281336 s, 36.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:28,426::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:28,453::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000754526 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:29,312::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:29,364::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278576 s, 36.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:29,655::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:29,656::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:33:29,767::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:33:29,768::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:33:29,769::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:33:29,769::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:31,370::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:31,418::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235313 s, 43.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:33,425::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:33,473::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227472 s, 45.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:35,480::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:35,520::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182891 s, 56.0 MB/s\n'; <rc> = 0
>Thread-1318::DEBUG::2012-12-09 12:33:37,245::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1318::DEBUG::2012-12-09 12:33:37,246::task::588::TaskManager.Task::(_updateState) Task=`2f5b3db8-718e-48d2-ada8-e92cb08b974a`::moving from state init -> state preparing
>Thread-1318::INFO::2012-12-09 12:33:37,247::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1318::INFO::2012-12-09 12:33:37,247::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1318::DEBUG::2012-12-09 12:33:37,248::task::1172::TaskManager.Task::(prepare) Task=`2f5b3db8-718e-48d2-ada8-e92cb08b974a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1318::DEBUG::2012-12-09 12:33:37,248::task::588::TaskManager.Task::(_updateState) Task=`2f5b3db8-718e-48d2-ada8-e92cb08b974a`::moving from state preparing -> state finished
>Thread-1318::DEBUG::2012-12-09 12:33:37,249::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1318::DEBUG::2012-12-09 12:33:37,249::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1318::DEBUG::2012-12-09 12:33:37,249::task::978::TaskManager.Task::(_decref) Task=`2f5b3db8-718e-48d2-ada8-e92cb08b974a`::ref 0 aborting False
>Thread-1319::DEBUG::2012-12-09 12:33:37,274::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1319::DEBUG::2012-12-09 12:33:37,274::task::588::TaskManager.Task::(_updateState) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::moving from state init -> state preparing
>Thread-1319::INFO::2012-12-09 12:33:37,275::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1319::DEBUG::2012-12-09 12:33:37,276::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`faddd8ce-b45b-4b98-b7cc-f6be3b26b939`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1319::DEBUG::2012-12-09 12:33:37,276::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1319::DEBUG::2012-12-09 12:33:37,276::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1319::DEBUG::2012-12-09 12:33:37,277::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`faddd8ce-b45b-4b98-b7cc-f6be3b26b939`::Granted request
>Thread-1319::DEBUG::2012-12-09 12:33:37,278::task::817::TaskManager.Task::(resourceAcquired) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1319::DEBUG::2012-12-09 12:33:37,278::task::978::TaskManager.Task::(_decref) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::ref 1 aborting False
>Thread-1319::DEBUG::2012-12-09 12:33:37,279::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1319::DEBUG::2012-12-09 12:33:37,280::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1320::DEBUG::2012-12-09 12:33:37,362::task::588::TaskManager.Task::(_updateState) Task=`12f78cfe-bf97-4fcc-a793-f0f8f56461f3`::moving from state init -> state preparing
>Thread-1320::INFO::2012-12-09 12:33:37,362::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1320::INFO::2012-12-09 12:33:37,362::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0280330181122', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-1320::DEBUG::2012-12-09 12:33:37,363::task::1172::TaskManager.Task::(prepare) Task=`12f78cfe-bf97-4fcc-a793-f0f8f56461f3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0280330181122', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-1320::DEBUG::2012-12-09 12:33:37,363::task::588::TaskManager.Task::(_updateState) Task=`12f78cfe-bf97-4fcc-a793-f0f8f56461f3`::moving from state preparing -> state finished
>Thread-1320::DEBUG::2012-12-09 12:33:37,363::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1320::DEBUG::2012-12-09 12:33:37,363::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1320::DEBUG::2012-12-09 12:33:37,363::task::978::TaskManager.Task::(_decref) Task=`12f78cfe-bf97-4fcc-a793-f0f8f56461f3`::ref 0 aborting False
>Thread-1319::DEBUG::2012-12-09 12:33:37,397::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1319::WARNING::2012-12-09 12:33:37,399::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1319::DEBUG::2012-12-09 12:33:37,399::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1319::WARNING::2012-12-09 12:33:37,400::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1319::INFO::2012-12-09 12:33:37,401::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1319::DEBUG::2012-12-09 12:33:37,401::task::1172::TaskManager.Task::(prepare) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1319::DEBUG::2012-12-09 12:33:37,401::task::588::TaskManager.Task::(_updateState) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::moving from state preparing -> state finished
>Thread-1319::DEBUG::2012-12-09 12:33:37,401::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1319::DEBUG::2012-12-09 12:33:37,402::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1319::DEBUG::2012-12-09 12:33:37,402::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1319::DEBUG::2012-12-09 12:33:37,402::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1319::DEBUG::2012-12-09 12:33:37,402::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1319::DEBUG::2012-12-09 12:33:37,403::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1319::DEBUG::2012-12-09 12:33:37,403::task::978::TaskManager.Task::(_decref) Task=`4f3c802d-8a2e-434a-a792-f14583c6388f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:37,527::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:37,577::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251612 s, 40.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:38,456::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:38,471::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000680326 s, 6.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:39,584::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:39,636::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268593 s, 38.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:39,770::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:39,772::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:33:39,874::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:33:39,875::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:33:39,876::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:33:39,876::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:41,645::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:41,700::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211812 s, 48.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:43,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:43,753::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258025 s, 39.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:45,759::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:45,808::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245429 s, 41.7 MB/s\n'; <rc> = 0
>Thread-1326::DEBUG::2012-12-09 12:33:47,663::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1326::DEBUG::2012-12-09 12:33:47,664::task::588::TaskManager.Task::(_updateState) Task=`3a55e50e-70cf-4066-b710-141b23d94b13`::moving from state init -> state preparing
>Thread-1326::INFO::2012-12-09 12:33:47,665::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1326::INFO::2012-12-09 12:33:47,665::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1326::DEBUG::2012-12-09 12:33:47,666::task::1172::TaskManager.Task::(prepare) Task=`3a55e50e-70cf-4066-b710-141b23d94b13`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1326::DEBUG::2012-12-09 12:33:47,666::task::588::TaskManager.Task::(_updateState) Task=`3a55e50e-70cf-4066-b710-141b23d94b13`::moving from state preparing -> state finished
>Thread-1326::DEBUG::2012-12-09 12:33:47,667::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1326::DEBUG::2012-12-09 12:33:47,667::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1326::DEBUG::2012-12-09 12:33:47,667::task::978::TaskManager.Task::(_decref) Task=`3a55e50e-70cf-4066-b710-141b23d94b13`::ref 0 aborting False
>Thread-1327::DEBUG::2012-12-09 12:33:47,687::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1327::DEBUG::2012-12-09 12:33:47,688::task::588::TaskManager.Task::(_updateState) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::moving from state init -> state preparing
>Thread-1327::INFO::2012-12-09 12:33:47,689::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1327::DEBUG::2012-12-09 12:33:47,691::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`adb36a18-5924-4e1c-9b70-27422f2a4daf`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1327::DEBUG::2012-12-09 12:33:47,691::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1327::DEBUG::2012-12-09 12:33:47,692::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1327::DEBUG::2012-12-09 12:33:47,692::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`adb36a18-5924-4e1c-9b70-27422f2a4daf`::Granted request
>Thread-1327::DEBUG::2012-12-09 12:33:47,693::task::817::TaskManager.Task::(resourceAcquired) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1327::DEBUG::2012-12-09 12:33:47,693::task::978::TaskManager.Task::(_decref) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::ref 1 aborting False
>Thread-1327::DEBUG::2012-12-09 12:33:47,694::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1327::DEBUG::2012-12-09 12:33:47,695::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1328::DEBUG::2012-12-09 12:33:47,778::task::588::TaskManager.Task::(_updateState) Task=`900cfdec-d9ef-461e-a3b3-9352ebe2398e`::moving from state init -> state preparing
>Thread-1328::INFO::2012-12-09 12:33:47,778::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1328::INFO::2012-12-09 12:33:47,778::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160751342773', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-1328::DEBUG::2012-12-09 12:33:47,779::task::1172::TaskManager.Task::(prepare) Task=`900cfdec-d9ef-461e-a3b3-9352ebe2398e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160751342773', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-1328::DEBUG::2012-12-09 12:33:47,779::task::588::TaskManager.Task::(_updateState) Task=`900cfdec-d9ef-461e-a3b3-9352ebe2398e`::moving from state preparing -> state finished
>Thread-1328::DEBUG::2012-12-09 12:33:47,779::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1328::DEBUG::2012-12-09 12:33:47,779::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1328::DEBUG::2012-12-09 12:33:47,779::task::978::TaskManager.Task::(_decref) Task=`900cfdec-d9ef-461e-a3b3-9352ebe2398e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:47,815::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1327::DEBUG::2012-12-09 12:33:47,822::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1327::WARNING::2012-12-09 12:33:47,824::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1327::DEBUG::2012-12-09 12:33:47,825::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1327::WARNING::2012-12-09 12:33:47,826::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1327::INFO::2012-12-09 12:33:47,827::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1327::DEBUG::2012-12-09 12:33:47,827::task::1172::TaskManager.Task::(prepare) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1327::DEBUG::2012-12-09 12:33:47,828::task::588::TaskManager.Task::(_updateState) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::moving from state preparing -> state finished
>Thread-1327::DEBUG::2012-12-09 12:33:47,828::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1327::DEBUG::2012-12-09 12:33:47,828::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1327::DEBUG::2012-12-09 12:33:47,829::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1327::DEBUG::2012-12-09 12:33:47,830::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1327::DEBUG::2012-12-09 12:33:47,830::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1327::DEBUG::2012-12-09 12:33:47,830::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1327::DEBUG::2012-12-09 12:33:47,831::task::978::TaskManager.Task::(_decref) Task=`f6e3c4e5-30f2-4433-9c65-eac328867484`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:47,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278299 s, 36.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:48,475::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:48,492::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000911496 s, 4.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:49,877::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:49,879::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:49,886::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:49,926::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016188 s, 63.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:49,990::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:33:49,992::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:33:49,992::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:33:49,993::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:33:51,932::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:51,988::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274402 s, 37.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:53,996::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:54,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262851 s, 39.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:33:56,057::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:33:56,107::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246777 s, 41.5 MB/s\n'; <rc> = 0
>Thread-1334::DEBUG::2012-12-09 12:33:57,985::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1334::DEBUG::2012-12-09 12:33:57,985::task::588::TaskManager.Task::(_updateState) Task=`b9b32410-9e4f-42d2-a665-095c33dab3d8`::moving from state init -> state preparing
>Thread-1334::INFO::2012-12-09 12:33:57,985::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1334::INFO::2012-12-09 12:33:57,986::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1334::DEBUG::2012-12-09 12:33:57,986::task::1172::TaskManager.Task::(prepare) Task=`b9b32410-9e4f-42d2-a665-095c33dab3d8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1334::DEBUG::2012-12-09 12:33:57,986::task::588::TaskManager.Task::(_updateState) Task=`b9b32410-9e4f-42d2-a665-095c33dab3d8`::moving from state preparing -> state finished
>Thread-1334::DEBUG::2012-12-09 12:33:57,986::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1334::DEBUG::2012-12-09 12:33:57,986::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1334::DEBUG::2012-12-09 12:33:57,986::task::978::TaskManager.Task::(_decref) Task=`b9b32410-9e4f-42d2-a665-095c33dab3d8`::ref 0 aborting False
>Thread-1335::DEBUG::2012-12-09 12:33:58,005::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1335::DEBUG::2012-12-09 12:33:58,006::task::588::TaskManager.Task::(_updateState) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::moving from state init -> state preparing
>Thread-1335::INFO::2012-12-09 12:33:58,007::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1335::DEBUG::2012-12-09 12:33:58,007::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`66f3d6ec-6f7e-4072-828d-dbcf50bea8ca`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1335::DEBUG::2012-12-09 12:33:58,008::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1335::DEBUG::2012-12-09 12:33:58,008::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1335::DEBUG::2012-12-09 12:33:58,009::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`66f3d6ec-6f7e-4072-828d-dbcf50bea8ca`::Granted request
>Thread-1335::DEBUG::2012-12-09 12:33:58,009::task::817::TaskManager.Task::(resourceAcquired) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1335::DEBUG::2012-12-09 12:33:58,010::task::978::TaskManager.Task::(_decref) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::ref 1 aborting False
>Thread-1335::DEBUG::2012-12-09 12:33:58,011::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1335::DEBUG::2012-12-09 12:33:58,012::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1336::DEBUG::2012-12-09 12:33:58,098::task::588::TaskManager.Task::(_updateState) Task=`fcb6cc12-7835-419d-80df-a1e1a9bbfca3`::moving from state init -> state preparing
>Thread-1336::INFO::2012-12-09 12:33:58,099::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1336::INFO::2012-12-09 12:33:58,099::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0172328948975', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-1336::DEBUG::2012-12-09 12:33:58,099::task::1172::TaskManager.Task::(prepare) Task=`fcb6cc12-7835-419d-80df-a1e1a9bbfca3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0172328948975', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-1336::DEBUG::2012-12-09 12:33:58,100::task::588::TaskManager.Task::(_updateState) Task=`fcb6cc12-7835-419d-80df-a1e1a9bbfca3`::moving from state preparing -> state finished
>Thread-1336::DEBUG::2012-12-09 12:33:58,100::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1336::DEBUG::2012-12-09 12:33:58,100::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1336::DEBUG::2012-12-09 12:33:58,100::task::978::TaskManager.Task::(_decref) Task=`fcb6cc12-7835-419d-80df-a1e1a9bbfca3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:58,113::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1335::DEBUG::2012-12-09 12:33:58,121::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1335::WARNING::2012-12-09 12:33:58,125::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1335::DEBUG::2012-12-09 12:33:58,126::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1335::WARNING::2012-12-09 12:33:58,128::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1335::INFO::2012-12-09 12:33:58,129::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1335::DEBUG::2012-12-09 12:33:58,130::task::1172::TaskManager.Task::(prepare) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1335::DEBUG::2012-12-09 12:33:58,130::task::588::TaskManager.Task::(_updateState) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::moving from state preparing -> state finished
>Thread-1335::DEBUG::2012-12-09 12:33:58,131::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1335::DEBUG::2012-12-09 12:33:58,132::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1335::DEBUG::2012-12-09 12:33:58,133::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1335::DEBUG::2012-12-09 12:33:58,134::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1335::DEBUG::2012-12-09 12:33:58,135::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1335::DEBUG::2012-12-09 12:33:58,135::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1335::DEBUG::2012-12-09 12:33:58,136::task::978::TaskManager.Task::(_decref) Task=`c2e127e5-26e7-4996-9e3d-97ec096cf3b7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:33:58,171::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0319714 s, 32.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:33:58,495::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:33:58,510::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000442545 s, 9.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:33:59,994::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:33:59,995::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:00,104::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:00,105::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:00,105::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:00,106::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:00,178::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:00,221::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188243 s, 54.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:02,227::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:02,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257593 s, 39.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:04,284::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:04,331::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022006 s, 46.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:06,337::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:06,387::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025171 s, 40.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:08,393::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1342::DEBUG::2012-12-09 12:34:08,402::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1342::DEBUG::2012-12-09 12:34:08,403::task::588::TaskManager.Task::(_updateState) Task=`2a0f4828-a736-4eed-ad2b-1b782922ad23`::moving from state init -> state preparing
>Thread-1342::INFO::2012-12-09 12:34:08,403::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1342::INFO::2012-12-09 12:34:08,403::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1342::DEBUG::2012-12-09 12:34:08,404::task::1172::TaskManager.Task::(prepare) Task=`2a0f4828-a736-4eed-ad2b-1b782922ad23`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1342::DEBUG::2012-12-09 12:34:08,404::task::588::TaskManager.Task::(_updateState) Task=`2a0f4828-a736-4eed-ad2b-1b782922ad23`::moving from state preparing -> state finished
>Thread-1342::DEBUG::2012-12-09 12:34:08,404::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1342::DEBUG::2012-12-09 12:34:08,404::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1342::DEBUG::2012-12-09 12:34:08,405::task::978::TaskManager.Task::(_decref) Task=`2a0f4828-a736-4eed-ad2b-1b782922ad23`::ref 0 aborting False
>Thread-1343::DEBUG::2012-12-09 12:34:08,422::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1343::DEBUG::2012-12-09 12:34:08,423::task::588::TaskManager.Task::(_updateState) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::moving from state init -> state preparing
>Thread-1343::INFO::2012-12-09 12:34:08,424::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1343::DEBUG::2012-12-09 12:34:08,425::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7dabe8f0-212b-420a-b4cd-eba001304b8b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1343::DEBUG::2012-12-09 12:34:08,426::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1343::DEBUG::2012-12-09 12:34:08,427::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1343::DEBUG::2012-12-09 12:34:08,428::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7dabe8f0-212b-420a-b4cd-eba001304b8b`::Granted request
>Thread-1343::DEBUG::2012-12-09 12:34:08,430::task::817::TaskManager.Task::(resourceAcquired) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1343::DEBUG::2012-12-09 12:34:08,430::task::978::TaskManager.Task::(_decref) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::ref 1 aborting False
>Thread-1343::DEBUG::2012-12-09 12:34:08,433::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1343::DEBUG::2012-12-09 12:34:08,434::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:08,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0403863 s, 25.4 MB/s\n'; <rc> = 0
>Thread-1344::DEBUG::2012-12-09 12:34:08,489::task::588::TaskManager.Task::(_updateState) Task=`cc1a8a17-5c2d-4f93-88b0-8e4426a07faf`::moving from state init -> state preparing
>Thread-1344::INFO::2012-12-09 12:34:08,489::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1344::INFO::2012-12-09 12:34:08,490::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160481929779', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-1344::DEBUG::2012-12-09 12:34:08,490::task::1172::TaskManager.Task::(prepare) Task=`cc1a8a17-5c2d-4f93-88b0-8e4426a07faf`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160481929779', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-1344::DEBUG::2012-12-09 12:34:08,490::task::588::TaskManager.Task::(_updateState) Task=`cc1a8a17-5c2d-4f93-88b0-8e4426a07faf`::moving from state preparing -> state finished
>Thread-1344::DEBUG::2012-12-09 12:34:08,491::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1344::DEBUG::2012-12-09 12:34:08,491::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1344::DEBUG::2012-12-09 12:34:08,491::task::978::TaskManager.Task::(_decref) Task=`cc1a8a17-5c2d-4f93-88b0-8e4426a07faf`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:34:08,514::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:34:08,514::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:34:08,515::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:08,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000354548 s, 5.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:08,530::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000354548 s, 5.8 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:34:08,530::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:34:08,531::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:34:08,531::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:34:08,531::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-1343::DEBUG::2012-12-09 12:34:08,544::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1343::WARNING::2012-12-09 12:34:08,545::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1343::DEBUG::2012-12-09 12:34:08,545::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:34:08,545::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-1343::WARNING::2012-12-09 12:34:08,547::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:34:08,547::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-1343::INFO::2012-12-09 12:34:08,548::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-50::DEBUG::2012-12-09 12:34:08,548::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-1343::DEBUG::2012-12-09 12:34:08,549::task::1172::TaskManager.Task::(prepare) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-50::DEBUG::2012-12-09 12:34:08,550::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-1343::DEBUG::2012-12-09 12:34:08,550::task::588::TaskManager.Task::(_updateState) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::moving from state preparing -> state finished
>Thread-50::DEBUG::2012-12-09 12:34:08,551::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1343::DEBUG::2012-12-09 12:34:08,551::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-50::DEBUG::2012-12-09 12:34:08,552::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-1343::DEBUG::2012-12-09 12:34:08,553::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1343::DEBUG::2012-12-09 12:34:08,559::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1343::DEBUG::2012-12-09 12:34:08,560::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1343::DEBUG::2012-12-09 12:34:08,560::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1343::DEBUG::2012-12-09 12:34:08,560::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1343::DEBUG::2012-12-09 12:34:08,560::task::978::TaskManager.Task::(_decref) Task=`4df86f0b-360f-4f63-9c92-77702b95d30a`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:34:08,657::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:08,659::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:34:08,660::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:34:08,661::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:34:08,662::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:34:08,662::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:34:08,663::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:08,765::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:08,766::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:08,781::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000957808 s, 4.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:34:10,107::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:34:10,108::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:10,223::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:10,224::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:10,224::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:10,225::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:10,468::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:10,529::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245431 s, 41.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:12,536::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:12,586::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269547 s, 38.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:14,592::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:14,639::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227565 s, 45.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:16,646::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:16,686::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166276 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:18,693::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1350::DEBUG::2012-12-09 12:34:18,718::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1350::DEBUG::2012-12-09 12:34:18,719::task::588::TaskManager.Task::(_updateState) Task=`ca1673e5-0df9-4e10-8f50-ff9a4dca33e2`::moving from state init -> state preparing
>Thread-1350::INFO::2012-12-09 12:34:18,719::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1350::INFO::2012-12-09 12:34:18,720::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1350::DEBUG::2012-12-09 12:34:18,720::task::1172::TaskManager.Task::(prepare) Task=`ca1673e5-0df9-4e10-8f50-ff9a4dca33e2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1350::DEBUG::2012-12-09 12:34:18,721::task::588::TaskManager.Task::(_updateState) Task=`ca1673e5-0df9-4e10-8f50-ff9a4dca33e2`::moving from state preparing -> state finished
>Thread-1350::DEBUG::2012-12-09 12:34:18,722::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1350::DEBUG::2012-12-09 12:34:18,722::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1350::DEBUG::2012-12-09 12:34:18,723::task::978::TaskManager.Task::(_decref) Task=`ca1673e5-0df9-4e10-8f50-ff9a4dca33e2`::ref 0 aborting False
>Thread-1351::DEBUG::2012-12-09 12:34:18,740::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1351::DEBUG::2012-12-09 12:34:18,741::task::588::TaskManager.Task::(_updateState) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::moving from state init -> state preparing
>Thread-1351::INFO::2012-12-09 12:34:18,741::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1351::DEBUG::2012-12-09 12:34:18,742::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`696de292-f6df-4332-ac6f-1a193e8ead9d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1351::DEBUG::2012-12-09 12:34:18,743::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1351::DEBUG::2012-12-09 12:34:18,744::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1351::DEBUG::2012-12-09 12:34:18,744::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`696de292-f6df-4332-ac6f-1a193e8ead9d`::Granted request
>Thread-1351::DEBUG::2012-12-09 12:34:18,745::task::817::TaskManager.Task::(resourceAcquired) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1351::DEBUG::2012-12-09 12:34:18,746::task::978::TaskManager.Task::(_decref) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::ref 1 aborting False
>Thread-1351::DEBUG::2012-12-09 12:34:18,748::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1351::DEBUG::2012-12-09 12:34:18,749::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:18,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0331876 s, 30.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:18,785::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:18,809::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000701153 s, 5.8 MB/s\n'; <rc> = 0
>Thread-1352::DEBUG::2012-12-09 12:34:18,841::task::588::TaskManager.Task::(_updateState) Task=`e4f42129-596c-48f2-9aff-584d48f2ebda`::moving from state init -> state preparing
>Thread-1352::INFO::2012-12-09 12:34:18,841::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1352::INFO::2012-12-09 12:34:18,841::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0243039131165', 'lastCheck': '0.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-1352::DEBUG::2012-12-09 12:34:18,842::task::1172::TaskManager.Task::(prepare) Task=`e4f42129-596c-48f2-9aff-584d48f2ebda`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0243039131165', 'lastCheck': '0.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-1352::DEBUG::2012-12-09 12:34:18,842::task::588::TaskManager.Task::(_updateState) Task=`e4f42129-596c-48f2-9aff-584d48f2ebda`::moving from state preparing -> state finished
>Thread-1352::DEBUG::2012-12-09 12:34:18,842::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1352::DEBUG::2012-12-09 12:34:18,842::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1352::DEBUG::2012-12-09 12:34:18,843::task::978::TaskManager.Task::(_decref) Task=`e4f42129-596c-48f2-9aff-584d48f2ebda`::ref 0 aborting False
>Thread-1351::DEBUG::2012-12-09 12:34:18,851::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1351::WARNING::2012-12-09 12:34:18,853::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1351::DEBUG::2012-12-09 12:34:18,853::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1351::WARNING::2012-12-09 12:34:18,854::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1351::INFO::2012-12-09 12:34:18,854::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1351::DEBUG::2012-12-09 12:34:18,854::task::1172::TaskManager.Task::(prepare) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1351::DEBUG::2012-12-09 12:34:18,854::task::588::TaskManager.Task::(_updateState) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::moving from state preparing -> state finished
>Thread-1351::DEBUG::2012-12-09 12:34:18,855::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1351::DEBUG::2012-12-09 12:34:18,855::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1351::DEBUG::2012-12-09 12:34:18,855::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1351::DEBUG::2012-12-09 12:34:18,855::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1351::DEBUG::2012-12-09 12:34:18,855::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1351::DEBUG::2012-12-09 12:34:18,856::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1351::DEBUG::2012-12-09 12:34:18,856::task::978::TaskManager.Task::(_decref) Task=`40eaad18-5262-4fba-85d4-c19cd99b6ec7`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:34:20,225::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:34:20,226::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:34:20,227::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:20,343::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:20,344::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:20,345::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:20,345::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:20,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:20,825::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260228 s, 39.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:22,832::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:22,879::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226415 s, 45.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:24,886::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:24,936::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264474 s, 38.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:26,943::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:26,990::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022619 s, 45.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:28,814::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:28,829::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000919106 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:28,997::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1358::DEBUG::2012-12-09 12:34:29,008::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1358::DEBUG::2012-12-09 12:34:29,011::task::588::TaskManager.Task::(_updateState) Task=`e47a4e10-657e-4188-8fec-94524d08ebb5`::moving from state init -> state preparing
>Thread-1358::INFO::2012-12-09 12:34:29,012::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1358::INFO::2012-12-09 12:34:29,013::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1358::DEBUG::2012-12-09 12:34:29,014::task::1172::TaskManager.Task::(prepare) Task=`e47a4e10-657e-4188-8fec-94524d08ebb5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1358::DEBUG::2012-12-09 12:34:29,016::task::588::TaskManager.Task::(_updateState) Task=`e47a4e10-657e-4188-8fec-94524d08ebb5`::moving from state preparing -> state finished
>Thread-1358::DEBUG::2012-12-09 12:34:29,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1358::DEBUG::2012-12-09 12:34:29,018::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1358::DEBUG::2012-12-09 12:34:29,020::task::978::TaskManager.Task::(_decref) Task=`e47a4e10-657e-4188-8fec-94524d08ebb5`::ref 0 aborting False
>Thread-1359::DEBUG::2012-12-09 12:34:29,046::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1359::DEBUG::2012-12-09 12:34:29,047::task::588::TaskManager.Task::(_updateState) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::moving from state init -> state preparing
>Thread-1359::INFO::2012-12-09 12:34:29,047::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1359::DEBUG::2012-12-09 12:34:29,048::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cab68b20-2dd9-4212-a416-6418ee4bee56`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1359::DEBUG::2012-12-09 12:34:29,049::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1359::DEBUG::2012-12-09 12:34:29,050::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1359::DEBUG::2012-12-09 12:34:29,050::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cab68b20-2dd9-4212-a416-6418ee4bee56`::Granted request
>Thread-1359::DEBUG::2012-12-09 12:34:29,051::task::817::TaskManager.Task::(resourceAcquired) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1359::DEBUG::2012-12-09 12:34:29,052::task::978::TaskManager.Task::(_decref) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::ref 1 aborting False
>Thread-1359::DEBUG::2012-12-09 12:34:29,053::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1359::DEBUG::2012-12-09 12:34:29,054::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:29,083::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0615846 s, 16.6 MB/s\n'; <rc> = 0
>Thread-1360::DEBUG::2012-12-09 12:34:29,134::task::588::TaskManager.Task::(_updateState) Task=`b1b09574-d12c-4a50-99cb-5810d84ba566`::moving from state init -> state preparing
>Thread-1360::INFO::2012-12-09 12:34:29,134::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1360::INFO::2012-12-09 12:34:29,134::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160310268402', 'lastCheck': '0.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-1360::DEBUG::2012-12-09 12:34:29,135::task::1172::TaskManager.Task::(prepare) Task=`b1b09574-d12c-4a50-99cb-5810d84ba566`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160310268402', 'lastCheck': '0.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-1360::DEBUG::2012-12-09 12:34:29,135::task::588::TaskManager.Task::(_updateState) Task=`b1b09574-d12c-4a50-99cb-5810d84ba566`::moving from state preparing -> state finished
>Thread-1360::DEBUG::2012-12-09 12:34:29,135::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1360::DEBUG::2012-12-09 12:34:29,135::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1360::DEBUG::2012-12-09 12:34:29,135::task::978::TaskManager.Task::(_decref) Task=`b1b09574-d12c-4a50-99cb-5810d84ba566`::ref 0 aborting False
>Thread-1359::DEBUG::2012-12-09 12:34:29,152::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1359::WARNING::2012-12-09 12:34:29,154::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1359::DEBUG::2012-12-09 12:34:29,155::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1359::WARNING::2012-12-09 12:34:29,155::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1359::INFO::2012-12-09 12:34:29,156::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1359::DEBUG::2012-12-09 12:34:29,157::task::1172::TaskManager.Task::(prepare) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1359::DEBUG::2012-12-09 12:34:29,157::task::588::TaskManager.Task::(_updateState) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::moving from state preparing -> state finished
>Thread-1359::DEBUG::2012-12-09 12:34:29,158::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1359::DEBUG::2012-12-09 12:34:29,158::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1359::DEBUG::2012-12-09 12:34:29,159::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1359::DEBUG::2012-12-09 12:34:29,159::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1359::DEBUG::2012-12-09 12:34:29,160::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1359::DEBUG::2012-12-09 12:34:29,160::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1359::DEBUG::2012-12-09 12:34:29,161::task::978::TaskManager.Task::(_decref) Task=`b236b73c-25c8-469a-a1e1-bbb3a696c0c2`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:34:30,346::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:34:30,347::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:30,463::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:30,464::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:30,465::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:30,465::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:31,090::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:31,138::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236573 s, 43.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:33,144::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:33,185::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171011 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:35,192::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:35,240::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244912 s, 41.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:37,246::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:37,296::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026674 s, 38.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:38,834::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:38,849::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000889638 s, 4.6 MB/s\n'; <rc> = 0
>Thread-1366::DEBUG::2012-12-09 12:34:39,287::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1366::DEBUG::2012-12-09 12:34:39,288::task::588::TaskManager.Task::(_updateState) Task=`0c0da6b4-d150-48ec-b34d-b6efb2d85bb8`::moving from state init -> state preparing
>Thread-1366::INFO::2012-12-09 12:34:39,289::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1366::INFO::2012-12-09 12:34:39,289::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1366::DEBUG::2012-12-09 12:34:39,290::task::1172::TaskManager.Task::(prepare) Task=`0c0da6b4-d150-48ec-b34d-b6efb2d85bb8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1366::DEBUG::2012-12-09 12:34:39,290::task::588::TaskManager.Task::(_updateState) Task=`0c0da6b4-d150-48ec-b34d-b6efb2d85bb8`::moving from state preparing -> state finished
>Thread-1366::DEBUG::2012-12-09 12:34:39,290::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1366::DEBUG::2012-12-09 12:34:39,291::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1366::DEBUG::2012-12-09 12:34:39,291::task::978::TaskManager.Task::(_decref) Task=`0c0da6b4-d150-48ec-b34d-b6efb2d85bb8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:34:39,303::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1367::DEBUG::2012-12-09 12:34:39,311::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1367::DEBUG::2012-12-09 12:34:39,313::task::588::TaskManager.Task::(_updateState) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::moving from state init -> state preparing
>Thread-1367::INFO::2012-12-09 12:34:39,315::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1367::DEBUG::2012-12-09 12:34:39,316::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d1d70849-b6e5-4ed3-aba2-7deeab4a51eb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1367::DEBUG::2012-12-09 12:34:39,317::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1367::DEBUG::2012-12-09 12:34:39,318::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1367::DEBUG::2012-12-09 12:34:39,318::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d1d70849-b6e5-4ed3-aba2-7deeab4a51eb`::Granted request
>Thread-1367::DEBUG::2012-12-09 12:34:39,319::task::817::TaskManager.Task::(resourceAcquired) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1367::DEBUG::2012-12-09 12:34:39,320::task::978::TaskManager.Task::(_decref) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::ref 1 aborting False
>Thread-1367::DEBUG::2012-12-09 12:34:39,323::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1367::DEBUG::2012-12-09 12:34:39,324::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:39,362::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037922 s, 27.0 MB/s\n'; <rc> = 0
>Thread-1368::DEBUG::2012-12-09 12:34:39,397::task::588::TaskManager.Task::(_updateState) Task=`e4353972-29dc-474e-b288-6cb0f649c3d5`::moving from state init -> state preparing
>Thread-1368::INFO::2012-12-09 12:34:39,398::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1368::INFO::2012-12-09 12:34:39,398::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162041187286', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-1368::DEBUG::2012-12-09 12:34:39,398::task::1172::TaskManager.Task::(prepare) Task=`e4353972-29dc-474e-b288-6cb0f649c3d5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162041187286', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-1368::DEBUG::2012-12-09 12:34:39,399::task::588::TaskManager.Task::(_updateState) Task=`e4353972-29dc-474e-b288-6cb0f649c3d5`::moving from state preparing -> state finished
>Thread-1368::DEBUG::2012-12-09 12:34:39,399::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1368::DEBUG::2012-12-09 12:34:39,399::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1368::DEBUG::2012-12-09 12:34:39,399::task::978::TaskManager.Task::(_decref) Task=`e4353972-29dc-474e-b288-6cb0f649c3d5`::ref 0 aborting False
>Thread-1367::DEBUG::2012-12-09 12:34:39,422::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1367::WARNING::2012-12-09 12:34:39,426::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1367::DEBUG::2012-12-09 12:34:39,426::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1367::WARNING::2012-12-09 12:34:39,427::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1367::INFO::2012-12-09 12:34:39,428::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1367::DEBUG::2012-12-09 12:34:39,428::task::1172::TaskManager.Task::(prepare) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1367::DEBUG::2012-12-09 12:34:39,429::task::588::TaskManager.Task::(_updateState) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::moving from state preparing -> state finished
>Thread-1367::DEBUG::2012-12-09 12:34:39,429::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1367::DEBUG::2012-12-09 12:34:39,429::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1367::DEBUG::2012-12-09 12:34:39,430::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1367::DEBUG::2012-12-09 12:34:39,430::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1367::DEBUG::2012-12-09 12:34:39,430::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1367::DEBUG::2012-12-09 12:34:39,431::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1367::DEBUG::2012-12-09 12:34:39,431::task::978::TaskManager.Task::(_decref) Task=`6ee2956d-1d61-49ae-9276-5e96f242aceb`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:34:40,466::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:34:40,467::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:40,574::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:40,575::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:40,575::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:40,576::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:41,369::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:41,418::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024281 s, 42.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:43,425::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:43,476::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267952 s, 38.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:45,482::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:45,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022616 s, 45.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:47,537::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:47,590::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274469 s, 37.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:48,854::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:48,869::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000994036 s, 4.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:49,597::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:49,642::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207002 s, 49.5 MB/s\n'; <rc> = 0
>Thread-1374::DEBUG::2012-12-09 12:34:49,697::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1374::DEBUG::2012-12-09 12:34:49,698::task::588::TaskManager.Task::(_updateState) Task=`4e75a548-4c6b-41f2-947d-ba0c405f032b`::moving from state init -> state preparing
>Thread-1374::INFO::2012-12-09 12:34:49,699::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1374::INFO::2012-12-09 12:34:49,700::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1374::DEBUG::2012-12-09 12:34:49,700::task::1172::TaskManager.Task::(prepare) Task=`4e75a548-4c6b-41f2-947d-ba0c405f032b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1374::DEBUG::2012-12-09 12:34:49,701::task::588::TaskManager.Task::(_updateState) Task=`4e75a548-4c6b-41f2-947d-ba0c405f032b`::moving from state preparing -> state finished
>Thread-1374::DEBUG::2012-12-09 12:34:49,701::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1374::DEBUG::2012-12-09 12:34:49,701::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1374::DEBUG::2012-12-09 12:34:49,702::task::978::TaskManager.Task::(_decref) Task=`4e75a548-4c6b-41f2-947d-ba0c405f032b`::ref 0 aborting False
>Thread-1375::DEBUG::2012-12-09 12:34:49,719::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1375::DEBUG::2012-12-09 12:34:49,720::task::588::TaskManager.Task::(_updateState) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::moving from state init -> state preparing
>Thread-1375::INFO::2012-12-09 12:34:49,720::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1375::DEBUG::2012-12-09 12:34:49,721::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5ad5d4b2-9be1-4765-8dde-dadfc8e5ab48`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1375::DEBUG::2012-12-09 12:34:49,721::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1375::DEBUG::2012-12-09 12:34:49,722::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1375::DEBUG::2012-12-09 12:34:49,722::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5ad5d4b2-9be1-4765-8dde-dadfc8e5ab48`::Granted request
>Thread-1375::DEBUG::2012-12-09 12:34:49,723::task::817::TaskManager.Task::(resourceAcquired) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1375::DEBUG::2012-12-09 12:34:49,723::task::978::TaskManager.Task::(_decref) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::ref 1 aborting False
>Thread-1375::DEBUG::2012-12-09 12:34:49,725::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1375::DEBUG::2012-12-09 12:34:49,726::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1376::DEBUG::2012-12-09 12:34:49,811::task::588::TaskManager.Task::(_updateState) Task=`c741ac8a-afd9-48bc-9a9a-fa353f3de35a`::moving from state init -> state preparing
>Thread-1376::INFO::2012-12-09 12:34:49,812::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1376::INFO::2012-12-09 12:34:49,812::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161080360413', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-1376::DEBUG::2012-12-09 12:34:49,812::task::1172::TaskManager.Task::(prepare) Task=`c741ac8a-afd9-48bc-9a9a-fa353f3de35a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161080360413', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-1376::DEBUG::2012-12-09 12:34:49,813::task::588::TaskManager.Task::(_updateState) Task=`c741ac8a-afd9-48bc-9a9a-fa353f3de35a`::moving from state preparing -> state finished
>Thread-1376::DEBUG::2012-12-09 12:34:49,813::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1376::DEBUG::2012-12-09 12:34:49,813::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1376::DEBUG::2012-12-09 12:34:49,813::task::978::TaskManager.Task::(_decref) Task=`c741ac8a-afd9-48bc-9a9a-fa353f3de35a`::ref 0 aborting False
>Thread-1375::DEBUG::2012-12-09 12:34:49,825::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1375::WARNING::2012-12-09 12:34:49,826::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1375::DEBUG::2012-12-09 12:34:49,826::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1375::WARNING::2012-12-09 12:34:49,827::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1375::INFO::2012-12-09 12:34:49,827::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1375::DEBUG::2012-12-09 12:34:49,828::task::1172::TaskManager.Task::(prepare) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1375::DEBUG::2012-12-09 12:34:49,829::task::588::TaskManager.Task::(_updateState) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::moving from state preparing -> state finished
>Thread-1375::DEBUG::2012-12-09 12:34:49,829::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1375::DEBUG::2012-12-09 12:34:49,831::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1375::DEBUG::2012-12-09 12:34:49,831::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1375::DEBUG::2012-12-09 12:34:49,831::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1375::DEBUG::2012-12-09 12:34:49,832::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1375::DEBUG::2012-12-09 12:34:49,832::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1375::DEBUG::2012-12-09 12:34:49,832::task::978::TaskManager.Task::(_decref) Task=`952bd71f-bf09-44b1-a86c-41cd6d45f085`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:34:50,577::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:34:50,578::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:34:50,683::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:34:50,684::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:34:50,684::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:34:50,685::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:34:51,648::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:51,691::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186273 s, 55.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:53,696::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:53,736::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169804 s, 60.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:55,743::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:55,798::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0288014 s, 35.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:57,804::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:57,856::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273116 s, 37.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:34:58,874::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:34:58,888::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000834063 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:34:59,862::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:34:59,910::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230082 s, 44.5 MB/s\n'; <rc> = 0
>Thread-1382::DEBUG::2012-12-09 12:35:00,025::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1382::DEBUG::2012-12-09 12:35:00,026::task::588::TaskManager.Task::(_updateState) Task=`53411d89-c1f9-4af5-a35d-b08c21fe90be`::moving from state init -> state preparing
>Thread-1382::INFO::2012-12-09 12:35:00,026::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1382::INFO::2012-12-09 12:35:00,027::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1382::DEBUG::2012-12-09 12:35:00,027::task::1172::TaskManager.Task::(prepare) Task=`53411d89-c1f9-4af5-a35d-b08c21fe90be`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1382::DEBUG::2012-12-09 12:35:00,028::task::588::TaskManager.Task::(_updateState) Task=`53411d89-c1f9-4af5-a35d-b08c21fe90be`::moving from state preparing -> state finished
>Thread-1382::DEBUG::2012-12-09 12:35:00,028::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1382::DEBUG::2012-12-09 12:35:00,028::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1382::DEBUG::2012-12-09 12:35:00,029::task::978::TaskManager.Task::(_decref) Task=`53411d89-c1f9-4af5-a35d-b08c21fe90be`::ref 0 aborting False
>Thread-1383::DEBUG::2012-12-09 12:35:00,047::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1383::DEBUG::2012-12-09 12:35:00,048::task::588::TaskManager.Task::(_updateState) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::moving from state init -> state preparing
>Thread-1383::INFO::2012-12-09 12:35:00,049::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1383::DEBUG::2012-12-09 12:35:00,049::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`09fd09d0-84b5-4a6c-a0d3-b7995bfc23f2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1383::DEBUG::2012-12-09 12:35:00,050::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1383::DEBUG::2012-12-09 12:35:00,050::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1383::DEBUG::2012-12-09 12:35:00,051::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`09fd09d0-84b5-4a6c-a0d3-b7995bfc23f2`::Granted request
>Thread-1383::DEBUG::2012-12-09 12:35:00,051::task::817::TaskManager.Task::(resourceAcquired) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1383::DEBUG::2012-12-09 12:35:00,052::task::978::TaskManager.Task::(_decref) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::ref 1 aborting False
>Thread-1383::DEBUG::2012-12-09 12:35:00,053::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1383::DEBUG::2012-12-09 12:35:00,053::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1384::DEBUG::2012-12-09 12:35:00,121::task::588::TaskManager.Task::(_updateState) Task=`0046b4f5-001f-4fdf-b4ff-aea1dc4cc5b5`::moving from state init -> state preparing
>Thread-1384::INFO::2012-12-09 12:35:00,121::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1384::INFO::2012-12-09 12:35:00,121::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149509906769', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-1384::DEBUG::2012-12-09 12:35:00,122::task::1172::TaskManager.Task::(prepare) Task=`0046b4f5-001f-4fdf-b4ff-aea1dc4cc5b5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149509906769', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-1384::DEBUG::2012-12-09 12:35:00,122::task::588::TaskManager.Task::(_updateState) Task=`0046b4f5-001f-4fdf-b4ff-aea1dc4cc5b5`::moving from state preparing -> state finished
>Thread-1384::DEBUG::2012-12-09 12:35:00,122::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1384::DEBUG::2012-12-09 12:35:00,122::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1384::DEBUG::2012-12-09 12:35:00,122::task::978::TaskManager.Task::(_decref) Task=`0046b4f5-001f-4fdf-b4ff-aea1dc4cc5b5`::ref 0 aborting False
>Thread-1383::DEBUG::2012-12-09 12:35:00,165::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1383::WARNING::2012-12-09 12:35:00,166::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1383::DEBUG::2012-12-09 12:35:00,166::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1383::WARNING::2012-12-09 12:35:00,166::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1383::INFO::2012-12-09 12:35:00,166::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1383::DEBUG::2012-12-09 12:35:00,167::task::1172::TaskManager.Task::(prepare) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1383::DEBUG::2012-12-09 12:35:00,167::task::588::TaskManager.Task::(_updateState) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::moving from state preparing -> state finished
>Thread-1383::DEBUG::2012-12-09 12:35:00,167::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1383::DEBUG::2012-12-09 12:35:00,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1383::DEBUG::2012-12-09 12:35:00,167::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1383::DEBUG::2012-12-09 12:35:00,168::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1383::DEBUG::2012-12-09 12:35:00,168::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1383::DEBUG::2012-12-09 12:35:00,168::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1383::DEBUG::2012-12-09 12:35:00,168::task::978::TaskManager.Task::(_decref) Task=`6c8aa980-9db9-4fe9-9ea2-167f36d6bd24`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:35:00,686::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:35:00,687::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:35:00,790::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:00,792::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:00,792::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:00,793::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:35:01,916::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:01,967::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261292 s, 39.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:03,973::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:04,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279735 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:06,032::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:06,081::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235551 s, 43.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:08,087::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:08,138::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264715 s, 38.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:35:08,894::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:08,907::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000960787 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:10,144::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:10,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028057 s, 36.5 MB/s\n'; <rc> = 0
>Thread-1390::DEBUG::2012-12-09 12:35:10,365::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1390::DEBUG::2012-12-09 12:35:10,365::task::588::TaskManager.Task::(_updateState) Task=`481e0729-7cd3-4fa2-a1c8-051803d8bdf2`::moving from state init -> state preparing
>Thread-1390::INFO::2012-12-09 12:35:10,366::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1390::INFO::2012-12-09 12:35:10,366::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1390::DEBUG::2012-12-09 12:35:10,366::task::1172::TaskManager.Task::(prepare) Task=`481e0729-7cd3-4fa2-a1c8-051803d8bdf2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1390::DEBUG::2012-12-09 12:35:10,366::task::588::TaskManager.Task::(_updateState) Task=`481e0729-7cd3-4fa2-a1c8-051803d8bdf2`::moving from state preparing -> state finished
>Thread-1390::DEBUG::2012-12-09 12:35:10,366::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1390::DEBUG::2012-12-09 12:35:10,367::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1390::DEBUG::2012-12-09 12:35:10,367::task::978::TaskManager.Task::(_decref) Task=`481e0729-7cd3-4fa2-a1c8-051803d8bdf2`::ref 0 aborting False
>Thread-1391::DEBUG::2012-12-09 12:35:10,385::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1391::DEBUG::2012-12-09 12:35:10,386::task::588::TaskManager.Task::(_updateState) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::moving from state init -> state preparing
>Thread-1391::INFO::2012-12-09 12:35:10,387::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1391::DEBUG::2012-12-09 12:35:10,387::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76c038e2-e199-4786-b9fe-aa43b1c0ed5c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1391::DEBUG::2012-12-09 12:35:10,388::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1391::DEBUG::2012-12-09 12:35:10,388::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1391::DEBUG::2012-12-09 12:35:10,389::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76c038e2-e199-4786-b9fe-aa43b1c0ed5c`::Granted request
>Thread-1391::DEBUG::2012-12-09 12:35:10,390::task::817::TaskManager.Task::(resourceAcquired) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1391::DEBUG::2012-12-09 12:35:10,390::task::978::TaskManager.Task::(_decref) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::ref 1 aborting False
>Thread-1391::DEBUG::2012-12-09 12:35:10,392::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1391::DEBUG::2012-12-09 12:35:10,392::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1391::DEBUG::2012-12-09 12:35:10,504::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1391::WARNING::2012-12-09 12:35:10,506::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1391::DEBUG::2012-12-09 12:35:10,507::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1391::WARNING::2012-12-09 12:35:10,508::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1391::INFO::2012-12-09 12:35:10,508::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1391::DEBUG::2012-12-09 12:35:10,509::task::1172::TaskManager.Task::(prepare) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1391::DEBUG::2012-12-09 12:35:10,509::task::588::TaskManager.Task::(_updateState) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::moving from state preparing -> state finished
>Thread-1391::DEBUG::2012-12-09 12:35:10,510::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1391::DEBUG::2012-12-09 12:35:10,510::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1391::DEBUG::2012-12-09 12:35:10,511::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1391::DEBUG::2012-12-09 12:35:10,511::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1391::DEBUG::2012-12-09 12:35:10,512::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1391::DEBUG::2012-12-09 12:35:10,512::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1391::DEBUG::2012-12-09 12:35:10,513::task::978::TaskManager.Task::(_decref) Task=`d72b7b73-a86a-42db-b562-046d220b9e09`::ref 0 aborting False
>Thread-1392::DEBUG::2012-12-09 12:35:10,540::task::588::TaskManager.Task::(_updateState) Task=`126d3fb5-cd22-4b62-80ef-977bff419994`::moving from state init -> state preparing
>Thread-1392::INFO::2012-12-09 12:35:10,541::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1392::INFO::2012-12-09 12:35:10,542::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145571231842', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1392::DEBUG::2012-12-09 12:35:10,542::task::1172::TaskManager.Task::(prepare) Task=`126d3fb5-cd22-4b62-80ef-977bff419994`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145571231842', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1392::DEBUG::2012-12-09 12:35:10,543::task::588::TaskManager.Task::(_updateState) Task=`126d3fb5-cd22-4b62-80ef-977bff419994`::moving from state preparing -> state finished
>Thread-1392::DEBUG::2012-12-09 12:35:10,543::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1392::DEBUG::2012-12-09 12:35:10,543::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1392::DEBUG::2012-12-09 12:35:10,543::task::978::TaskManager.Task::(_decref) Task=`126d3fb5-cd22-4b62-80ef-977bff419994`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:35:10,794::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:35:10,794::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:35:10,902::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:10,904::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:10,904::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:10,904::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:35:12,203::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:12,250::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021578 s, 47.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:14,256::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:14,308::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272712 s, 37.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:16,314::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:16,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275522 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:18,372::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:18,419::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234472 s, 43.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:35:18,913::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:18,928::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000748365 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:20,426::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:20,469::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192371 s, 53.2 MB/s\n'; <rc> = 0
>Thread-1398::DEBUG::2012-12-09 12:35:20,641::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1398::DEBUG::2012-12-09 12:35:20,643::task::588::TaskManager.Task::(_updateState) Task=`9e4b2b02-1399-4d28-a349-968945bb2d1f`::moving from state init -> state preparing
>Thread-1398::INFO::2012-12-09 12:35:20,643::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1398::INFO::2012-12-09 12:35:20,644::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1398::DEBUG::2012-12-09 12:35:20,644::task::1172::TaskManager.Task::(prepare) Task=`9e4b2b02-1399-4d28-a349-968945bb2d1f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1398::DEBUG::2012-12-09 12:35:20,645::task::588::TaskManager.Task::(_updateState) Task=`9e4b2b02-1399-4d28-a349-968945bb2d1f`::moving from state preparing -> state finished
>Thread-1398::DEBUG::2012-12-09 12:35:20,645::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1398::DEBUG::2012-12-09 12:35:20,645::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1398::DEBUG::2012-12-09 12:35:20,646::task::978::TaskManager.Task::(_decref) Task=`9e4b2b02-1399-4d28-a349-968945bb2d1f`::ref 0 aborting False
>Thread-1399::DEBUG::2012-12-09 12:35:20,667::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1399::DEBUG::2012-12-09 12:35:20,667::task::588::TaskManager.Task::(_updateState) Task=`d85006f7-6543-4617-8245-3f604949c57a`::moving from state init -> state preparing
>Thread-1399::INFO::2012-12-09 12:35:20,668::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1399::DEBUG::2012-12-09 12:35:20,668::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6f6fd45-9f16-476b-98f9-2adb2e676474`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1399::DEBUG::2012-12-09 12:35:20,668::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1399::DEBUG::2012-12-09 12:35:20,668::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1399::DEBUG::2012-12-09 12:35:20,669::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6f6fd45-9f16-476b-98f9-2adb2e676474`::Granted request
>Thread-1399::DEBUG::2012-12-09 12:35:20,669::task::817::TaskManager.Task::(resourceAcquired) Task=`d85006f7-6543-4617-8245-3f604949c57a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1399::DEBUG::2012-12-09 12:35:20,669::task::978::TaskManager.Task::(_decref) Task=`d85006f7-6543-4617-8245-3f604949c57a`::ref 1 aborting False
>Thread-1399::DEBUG::2012-12-09 12:35:20,670::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1399::DEBUG::2012-12-09 12:35:20,670::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1399::DEBUG::2012-12-09 12:35:20,780::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1399::WARNING::2012-12-09 12:35:20,784::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1399::DEBUG::2012-12-09 12:35:20,784::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1399::WARNING::2012-12-09 12:35:20,785::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1399::INFO::2012-12-09 12:35:20,785::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1399::DEBUG::2012-12-09 12:35:20,785::task::1172::TaskManager.Task::(prepare) Task=`d85006f7-6543-4617-8245-3f604949c57a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1399::DEBUG::2012-12-09 12:35:20,786::task::588::TaskManager.Task::(_updateState) Task=`d85006f7-6543-4617-8245-3f604949c57a`::moving from state preparing -> state finished
>Thread-1399::DEBUG::2012-12-09 12:35:20,786::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1399::DEBUG::2012-12-09 12:35:20,786::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1399::DEBUG::2012-12-09 12:35:20,786::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1399::DEBUG::2012-12-09 12:35:20,787::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1399::DEBUG::2012-12-09 12:35:20,787::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1399::DEBUG::2012-12-09 12:35:20,787::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1399::DEBUG::2012-12-09 12:35:20,787::task::978::TaskManager.Task::(_decref) Task=`d85006f7-6543-4617-8245-3f604949c57a`::ref 0 aborting False
>Thread-1400::DEBUG::2012-12-09 12:35:20,793::task::588::TaskManager.Task::(_updateState) Task=`d4a2cc51-d29f-4d95-9f5f-6d7ec0484bad`::moving from state init -> state preparing
>Thread-1400::INFO::2012-12-09 12:35:20,793::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1400::INFO::2012-12-09 12:35:20,794::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162980556488', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-1400::DEBUG::2012-12-09 12:35:20,794::task::1172::TaskManager.Task::(prepare) Task=`d4a2cc51-d29f-4d95-9f5f-6d7ec0484bad`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162980556488', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-1400::DEBUG::2012-12-09 12:35:20,794::task::588::TaskManager.Task::(_updateState) Task=`d4a2cc51-d29f-4d95-9f5f-6d7ec0484bad`::moving from state preparing -> state finished
>Thread-1400::DEBUG::2012-12-09 12:35:20,794::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1400::DEBUG::2012-12-09 12:35:20,795::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1400::DEBUG::2012-12-09 12:35:20,795::task::978::TaskManager.Task::(_decref) Task=`d4a2cc51-d29f-4d95-9f5f-6d7ec0484bad`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:35:20,905::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:35:20,907::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:35:21,017::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:21,018::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:21,018::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:21,019::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:35:22,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:22,527::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026961 s, 38.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:24,534::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:24,587::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282833 s, 36.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:26,594::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:26,645::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256565 s, 39.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:28,652::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:28,700::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233548 s, 43.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:35:28,934::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:28,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000838604 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:30,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:30,758::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026562 s, 38.6 MB/s\n'; <rc> = 0
>Thread-1406::DEBUG::2012-12-09 12:35:30,946::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1406::DEBUG::2012-12-09 12:35:30,946::task::588::TaskManager.Task::(_updateState) Task=`fde1686c-e0bb-42d6-9cc6-ead9706647cf`::moving from state init -> state preparing
>Thread-1406::INFO::2012-12-09 12:35:30,946::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1406::INFO::2012-12-09 12:35:30,947::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1406::DEBUG::2012-12-09 12:35:30,947::task::1172::TaskManager.Task::(prepare) Task=`fde1686c-e0bb-42d6-9cc6-ead9706647cf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1406::DEBUG::2012-12-09 12:35:30,947::task::588::TaskManager.Task::(_updateState) Task=`fde1686c-e0bb-42d6-9cc6-ead9706647cf`::moving from state preparing -> state finished
>Thread-1406::DEBUG::2012-12-09 12:35:30,947::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1406::DEBUG::2012-12-09 12:35:30,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1406::DEBUG::2012-12-09 12:35:30,948::task::978::TaskManager.Task::(_decref) Task=`fde1686c-e0bb-42d6-9cc6-ead9706647cf`::ref 0 aborting False
>Thread-1407::DEBUG::2012-12-09 12:35:30,967::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1407::DEBUG::2012-12-09 12:35:30,968::task::588::TaskManager.Task::(_updateState) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::moving from state init -> state preparing
>Thread-1407::INFO::2012-12-09 12:35:30,968::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1407::DEBUG::2012-12-09 12:35:30,969::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0624c377-0f8c-4682-b049-637dca0b1995`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1407::DEBUG::2012-12-09 12:35:30,970::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1407::DEBUG::2012-12-09 12:35:30,970::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1407::DEBUG::2012-12-09 12:35:30,970::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0624c377-0f8c-4682-b049-637dca0b1995`::Granted request
>Thread-1407::DEBUG::2012-12-09 12:35:30,971::task::817::TaskManager.Task::(resourceAcquired) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1407::DEBUG::2012-12-09 12:35:30,972::task::978::TaskManager.Task::(_decref) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::ref 1 aborting False
>Thread-1407::DEBUG::2012-12-09 12:35:30,973::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1407::DEBUG::2012-12-09 12:35:30,973::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1407::DEBUG::2012-12-09 12:35:31,072::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1407::WARNING::2012-12-09 12:35:31,073::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1407::DEBUG::2012-12-09 12:35:31,074::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1407::WARNING::2012-12-09 12:35:31,076::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1407::INFO::2012-12-09 12:35:31,077::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-51::DEBUG::2012-12-09 12:35:31,077::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1407::DEBUG::2012-12-09 12:35:31,078::task::1172::TaskManager.Task::(prepare) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-51::DEBUG::2012-12-09 12:35:31,080::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1407::DEBUG::2012-12-09 12:35:31,081::task::588::TaskManager.Task::(_updateState) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::moving from state preparing -> state finished
>Thread-1407::DEBUG::2012-12-09 12:35:31,088::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1407::DEBUG::2012-12-09 12:35:31,088::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1407::DEBUG::2012-12-09 12:35:31,088::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1407::DEBUG::2012-12-09 12:35:31,089::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1407::DEBUG::2012-12-09 12:35:31,089::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1407::DEBUG::2012-12-09 12:35:31,089::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1407::DEBUG::2012-12-09 12:35:31,090::task::978::TaskManager.Task::(_decref) Task=`167159d7-aee5-4bbb-b7f8-6b519b80a740`::ref 0 aborting False
>Thread-1408::DEBUG::2012-12-09 12:35:31,102::task::588::TaskManager.Task::(_updateState) Task=`c779282f-4893-4c59-afad-e56841167f62`::moving from state init -> state preparing
>Thread-1408::INFO::2012-12-09 12:35:31,103::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1408::INFO::2012-12-09 12:35:31,103::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164170265198', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-1408::DEBUG::2012-12-09 12:35:31,103::task::1172::TaskManager.Task::(prepare) Task=`c779282f-4893-4c59-afad-e56841167f62`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164170265198', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-1408::DEBUG::2012-12-09 12:35:31,103::task::588::TaskManager.Task::(_updateState) Task=`c779282f-4893-4c59-afad-e56841167f62`::moving from state preparing -> state finished
>Thread-1408::DEBUG::2012-12-09 12:35:31,104::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1408::DEBUG::2012-12-09 12:35:31,104::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1408::DEBUG::2012-12-09 12:35:31,104::task::978::TaskManager.Task::(_decref) Task=`c779282f-4893-4c59-afad-e56841167f62`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:35:31,186::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:31,188::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:31,188::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:31,189::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:35:32,764::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:32,815::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271534 s, 37.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:34,822::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:34,868::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023311 s, 43.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:36,875::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:36,922::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218444 s, 46.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:38,929::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:38,955::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:38,968::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000380916 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:38,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0434524 s, 23.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:41,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:41,051::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021593 s, 47.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:35:41,190::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:35:41,192::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:35:41,297::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:41,299::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:41,299::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:41,300::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1414::DEBUG::2012-12-09 12:35:41,326::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1414::DEBUG::2012-12-09 12:35:41,327::task::588::TaskManager.Task::(_updateState) Task=`163025f9-74b3-417e-8922-9f547872c78b`::moving from state init -> state preparing
>Thread-1414::INFO::2012-12-09 12:35:41,328::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1414::INFO::2012-12-09 12:35:41,329::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1414::DEBUG::2012-12-09 12:35:41,329::task::1172::TaskManager.Task::(prepare) Task=`163025f9-74b3-417e-8922-9f547872c78b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1414::DEBUG::2012-12-09 12:35:41,329::task::588::TaskManager.Task::(_updateState) Task=`163025f9-74b3-417e-8922-9f547872c78b`::moving from state preparing -> state finished
>Thread-1414::DEBUG::2012-12-09 12:35:41,330::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1414::DEBUG::2012-12-09 12:35:41,330::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1414::DEBUG::2012-12-09 12:35:41,331::task::978::TaskManager.Task::(_decref) Task=`163025f9-74b3-417e-8922-9f547872c78b`::ref 0 aborting False
>Thread-1415::DEBUG::2012-12-09 12:35:41,349::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1415::DEBUG::2012-12-09 12:35:41,350::task::588::TaskManager.Task::(_updateState) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::moving from state init -> state preparing
>Thread-1415::INFO::2012-12-09 12:35:41,351::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1415::DEBUG::2012-12-09 12:35:41,351::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`684d8620-6fbb-4225-843d-7cefffaa113f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1415::DEBUG::2012-12-09 12:35:41,352::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1415::DEBUG::2012-12-09 12:35:41,352::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1415::DEBUG::2012-12-09 12:35:41,353::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`684d8620-6fbb-4225-843d-7cefffaa113f`::Granted request
>Thread-1415::DEBUG::2012-12-09 12:35:41,353::task::817::TaskManager.Task::(resourceAcquired) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1415::DEBUG::2012-12-09 12:35:41,353::task::978::TaskManager.Task::(_decref) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::ref 1 aborting False
>Thread-1415::DEBUG::2012-12-09 12:35:41,354::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1415::DEBUG::2012-12-09 12:35:41,354::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1415::DEBUG::2012-12-09 12:35:41,456::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1415::WARNING::2012-12-09 12:35:41,457::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1415::DEBUG::2012-12-09 12:35:41,457::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1415::WARNING::2012-12-09 12:35:41,458::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1415::INFO::2012-12-09 12:35:41,458::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1415::DEBUG::2012-12-09 12:35:41,458::task::1172::TaskManager.Task::(prepare) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1415::DEBUG::2012-12-09 12:35:41,458::task::588::TaskManager.Task::(_updateState) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::moving from state preparing -> state finished
>Thread-1415::DEBUG::2012-12-09 12:35:41,459::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1415::DEBUG::2012-12-09 12:35:41,459::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1415::DEBUG::2012-12-09 12:35:41,459::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1415::DEBUG::2012-12-09 12:35:41,459::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1415::DEBUG::2012-12-09 12:35:41,460::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1415::DEBUG::2012-12-09 12:35:41,460::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1415::DEBUG::2012-12-09 12:35:41,460::task::978::TaskManager.Task::(_decref) Task=`5dfa8ef1-4195-48ba-9047-64493920ce00`::ref 0 aborting False
>Thread-1416::DEBUG::2012-12-09 12:35:41,465::task::588::TaskManager.Task::(_updateState) Task=`e4ca32e1-91db-4dcc-8a10-2b98089aa09e`::moving from state init -> state preparing
>Thread-1416::INFO::2012-12-09 12:35:41,465::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1416::INFO::2012-12-09 12:35:41,466::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141570568085', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1416::DEBUG::2012-12-09 12:35:41,466::task::1172::TaskManager.Task::(prepare) Task=`e4ca32e1-91db-4dcc-8a10-2b98089aa09e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141570568085', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1416::DEBUG::2012-12-09 12:35:41,466::task::588::TaskManager.Task::(_updateState) Task=`e4ca32e1-91db-4dcc-8a10-2b98089aa09e`::moving from state preparing -> state finished
>Thread-1416::DEBUG::2012-12-09 12:35:41,467::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1416::DEBUG::2012-12-09 12:35:41,467::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1416::DEBUG::2012-12-09 12:35:41,467::task::978::TaskManager.Task::(_decref) Task=`e4ca32e1-91db-4dcc-8a10-2b98089aa09e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:35:43,058::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:43,108::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256411 s, 39.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:45,114::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:45,171::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322856 s, 31.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:47,177::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:47,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167341 s, 61.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:35:48,974::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:48,990::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00081456 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:49,225::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:49,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270946 s, 37.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:51,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:35:51,301::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:35:51,304::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:51,347::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0393359 s, 26.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:35:51,414::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:35:51,415::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:35:51,416::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:35:51,417::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1422::DEBUG::2012-12-09 12:35:51,636::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1422::DEBUG::2012-12-09 12:35:51,637::task::588::TaskManager.Task::(_updateState) Task=`54edd2ec-9819-4975-8b38-ca51fd44d40a`::moving from state init -> state preparing
>Thread-1422::INFO::2012-12-09 12:35:51,637::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1422::INFO::2012-12-09 12:35:51,638::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1422::DEBUG::2012-12-09 12:35:51,638::task::1172::TaskManager.Task::(prepare) Task=`54edd2ec-9819-4975-8b38-ca51fd44d40a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1422::DEBUG::2012-12-09 12:35:51,638::task::588::TaskManager.Task::(_updateState) Task=`54edd2ec-9819-4975-8b38-ca51fd44d40a`::moving from state preparing -> state finished
>Thread-1422::DEBUG::2012-12-09 12:35:51,639::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1422::DEBUG::2012-12-09 12:35:51,639::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1422::DEBUG::2012-12-09 12:35:51,639::task::978::TaskManager.Task::(_decref) Task=`54edd2ec-9819-4975-8b38-ca51fd44d40a`::ref 0 aborting False
>Thread-1423::DEBUG::2012-12-09 12:35:51,654::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1423::DEBUG::2012-12-09 12:35:51,654::task::588::TaskManager.Task::(_updateState) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::moving from state init -> state preparing
>Thread-1423::INFO::2012-12-09 12:35:51,655::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1423::DEBUG::2012-12-09 12:35:51,655::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8123b64b-f086-4c12-81e7-4d12b78c1e91`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1423::DEBUG::2012-12-09 12:35:51,655::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1423::DEBUG::2012-12-09 12:35:51,656::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1423::DEBUG::2012-12-09 12:35:51,656::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8123b64b-f086-4c12-81e7-4d12b78c1e91`::Granted request
>Thread-1423::DEBUG::2012-12-09 12:35:51,657::task::817::TaskManager.Task::(resourceAcquired) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1423::DEBUG::2012-12-09 12:35:51,658::task::978::TaskManager.Task::(_decref) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::ref 1 aborting False
>Thread-1423::DEBUG::2012-12-09 12:35:51,659::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1423::DEBUG::2012-12-09 12:35:51,660::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1424::DEBUG::2012-12-09 12:35:51,734::task::588::TaskManager.Task::(_updateState) Task=`b8ab611c-10e5-4d8e-b061-29643964791b`::moving from state init -> state preparing
>Thread-1424::INFO::2012-12-09 12:35:51,734::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1424::INFO::2012-12-09 12:35:51,735::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169100761414', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-1424::DEBUG::2012-12-09 12:35:51,735::task::1172::TaskManager.Task::(prepare) Task=`b8ab611c-10e5-4d8e-b061-29643964791b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169100761414', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-1424::DEBUG::2012-12-09 12:35:51,735::task::588::TaskManager.Task::(_updateState) Task=`b8ab611c-10e5-4d8e-b061-29643964791b`::moving from state preparing -> state finished
>Thread-1424::DEBUG::2012-12-09 12:35:51,735::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1424::DEBUG::2012-12-09 12:35:51,736::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1424::DEBUG::2012-12-09 12:35:51,736::task::978::TaskManager.Task::(_decref) Task=`b8ab611c-10e5-4d8e-b061-29643964791b`::ref 0 aborting False
>Thread-1423::DEBUG::2012-12-09 12:35:51,760::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1423::WARNING::2012-12-09 12:35:51,761::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1423::DEBUG::2012-12-09 12:35:51,761::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1423::WARNING::2012-12-09 12:35:51,762::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1423::INFO::2012-12-09 12:35:51,762::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1423::DEBUG::2012-12-09 12:35:51,762::task::1172::TaskManager.Task::(prepare) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1423::DEBUG::2012-12-09 12:35:51,763::task::588::TaskManager.Task::(_updateState) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::moving from state preparing -> state finished
>Thread-1423::DEBUG::2012-12-09 12:35:51,763::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1423::DEBUG::2012-12-09 12:35:51,763::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1423::DEBUG::2012-12-09 12:35:51,763::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1423::DEBUG::2012-12-09 12:35:51,763::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1423::DEBUG::2012-12-09 12:35:51,764::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1423::DEBUG::2012-12-09 12:35:51,764::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1423::DEBUG::2012-12-09 12:35:51,764::task::978::TaskManager.Task::(_decref) Task=`d386bdf2-d31c-45b7-b5ef-b5a80904463f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:35:53,354::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:53,401::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218534 s, 46.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:55,407::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:55,457::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025524 s, 40.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:57,464::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:57,509::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212914 s, 48.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:35:58,995::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:35:59,011::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00092448 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:35:59,516::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:35:59,568::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266544 s, 38.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:01,418::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:01,419::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:01,534::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:01,535::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:01,535::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:01,535::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:01,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:01,622::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213907 s, 47.9 MB/s\n'; <rc> = 0
>Thread-1430::DEBUG::2012-12-09 12:36:01,945::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1430::DEBUG::2012-12-09 12:36:01,946::task::588::TaskManager.Task::(_updateState) Task=`9170465c-063a-4a91-af24-8ab1882611f6`::moving from state init -> state preparing
>Thread-1430::INFO::2012-12-09 12:36:01,946::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1430::INFO::2012-12-09 12:36:01,947::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1430::DEBUG::2012-12-09 12:36:01,947::task::1172::TaskManager.Task::(prepare) Task=`9170465c-063a-4a91-af24-8ab1882611f6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1430::DEBUG::2012-12-09 12:36:01,947::task::588::TaskManager.Task::(_updateState) Task=`9170465c-063a-4a91-af24-8ab1882611f6`::moving from state preparing -> state finished
>Thread-1430::DEBUG::2012-12-09 12:36:01,947::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1430::DEBUG::2012-12-09 12:36:01,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1430::DEBUG::2012-12-09 12:36:01,947::task::978::TaskManager.Task::(_decref) Task=`9170465c-063a-4a91-af24-8ab1882611f6`::ref 0 aborting False
>Thread-1431::DEBUG::2012-12-09 12:36:01,968::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1431::DEBUG::2012-12-09 12:36:01,968::task::588::TaskManager.Task::(_updateState) Task=`a387c958-6a7e-4946-b336-19f384703d99`::moving from state init -> state preparing
>Thread-1431::INFO::2012-12-09 12:36:01,969::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1431::DEBUG::2012-12-09 12:36:01,970::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c2a033ec-5287-4188-8156-4649dedff40f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1431::DEBUG::2012-12-09 12:36:01,970::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1431::DEBUG::2012-12-09 12:36:01,971::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1431::DEBUG::2012-12-09 12:36:01,971::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c2a033ec-5287-4188-8156-4649dedff40f`::Granted request
>Thread-1431::DEBUG::2012-12-09 12:36:01,972::task::817::TaskManager.Task::(resourceAcquired) Task=`a387c958-6a7e-4946-b336-19f384703d99`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1431::DEBUG::2012-12-09 12:36:01,972::task::978::TaskManager.Task::(_decref) Task=`a387c958-6a7e-4946-b336-19f384703d99`::ref 1 aborting False
>Thread-1431::DEBUG::2012-12-09 12:36:01,973::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1431::DEBUG::2012-12-09 12:36:01,974::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1432::DEBUG::2012-12-09 12:36:02,038::task::588::TaskManager.Task::(_updateState) Task=`5475f6a3-a435-4a29-b787-e762a1f89e58`::moving from state init -> state preparing
>Thread-1432::INFO::2012-12-09 12:36:02,039::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1432::INFO::2012-12-09 12:36:02,039::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169529914856', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-1432::DEBUG::2012-12-09 12:36:02,040::task::1172::TaskManager.Task::(prepare) Task=`5475f6a3-a435-4a29-b787-e762a1f89e58`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169529914856', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-1432::DEBUG::2012-12-09 12:36:02,040::task::588::TaskManager.Task::(_updateState) Task=`5475f6a3-a435-4a29-b787-e762a1f89e58`::moving from state preparing -> state finished
>Thread-1432::DEBUG::2012-12-09 12:36:02,040::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1432::DEBUG::2012-12-09 12:36:02,041::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1432::DEBUG::2012-12-09 12:36:02,041::task::978::TaskManager.Task::(_decref) Task=`5475f6a3-a435-4a29-b787-e762a1f89e58`::ref 0 aborting False
>Thread-1431::DEBUG::2012-12-09 12:36:02,089::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1431::WARNING::2012-12-09 12:36:02,091::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1431::DEBUG::2012-12-09 12:36:02,092::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1431::WARNING::2012-12-09 12:36:02,093::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1431::INFO::2012-12-09 12:36:02,093::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1431::DEBUG::2012-12-09 12:36:02,093::task::1172::TaskManager.Task::(prepare) Task=`a387c958-6a7e-4946-b336-19f384703d99`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1431::DEBUG::2012-12-09 12:36:02,094::task::588::TaskManager.Task::(_updateState) Task=`a387c958-6a7e-4946-b336-19f384703d99`::moving from state preparing -> state finished
>Thread-1431::DEBUG::2012-12-09 12:36:02,094::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1431::DEBUG::2012-12-09 12:36:02,094::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1431::DEBUG::2012-12-09 12:36:02,094::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1431::DEBUG::2012-12-09 12:36:02,095::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1431::DEBUG::2012-12-09 12:36:02,095::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1431::DEBUG::2012-12-09 12:36:02,095::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1431::DEBUG::2012-12-09 12:36:02,095::task::978::TaskManager.Task::(_decref) Task=`a387c958-6a7e-4946-b336-19f384703d99`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:03,629::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:03,673::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190942 s, 53.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:05,680::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:05,724::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195784 s, 52.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:07,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:07,782::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230036 s, 44.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:09,015::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:09,031::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000841208 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:09,789::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:09,845::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243048 s, 42.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:11,536::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:11,537::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:11,653::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:11,654::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:11,655::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:11,656::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:11,852::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:11,894::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192181 s, 53.3 MB/s\n'; <rc> = 0
>Thread-1438::DEBUG::2012-12-09 12:36:12,241::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1438::DEBUG::2012-12-09 12:36:12,241::task::588::TaskManager.Task::(_updateState) Task=`9ae43498-80a7-48f9-aebb-572e3a11a8eb`::moving from state init -> state preparing
>Thread-1438::INFO::2012-12-09 12:36:12,242::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1438::INFO::2012-12-09 12:36:12,242::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1438::DEBUG::2012-12-09 12:36:12,242::task::1172::TaskManager.Task::(prepare) Task=`9ae43498-80a7-48f9-aebb-572e3a11a8eb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1438::DEBUG::2012-12-09 12:36:12,242::task::588::TaskManager.Task::(_updateState) Task=`9ae43498-80a7-48f9-aebb-572e3a11a8eb`::moving from state preparing -> state finished
>Thread-1438::DEBUG::2012-12-09 12:36:12,242::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1438::DEBUG::2012-12-09 12:36:12,243::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1438::DEBUG::2012-12-09 12:36:12,243::task::978::TaskManager.Task::(_decref) Task=`9ae43498-80a7-48f9-aebb-572e3a11a8eb`::ref 0 aborting False
>Thread-1439::DEBUG::2012-12-09 12:36:12,266::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1439::DEBUG::2012-12-09 12:36:12,267::task::588::TaskManager.Task::(_updateState) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::moving from state init -> state preparing
>Thread-1439::INFO::2012-12-09 12:36:12,268::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1439::DEBUG::2012-12-09 12:36:12,268::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1b2fee7-3653-4be8-a0a9-8db19b85278d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1439::DEBUG::2012-12-09 12:36:12,269::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1439::DEBUG::2012-12-09 12:36:12,269::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1439::DEBUG::2012-12-09 12:36:12,270::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1b2fee7-3653-4be8-a0a9-8db19b85278d`::Granted request
>Thread-1439::DEBUG::2012-12-09 12:36:12,270::task::817::TaskManager.Task::(resourceAcquired) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1439::DEBUG::2012-12-09 12:36:12,271::task::978::TaskManager.Task::(_decref) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::ref 1 aborting False
>Thread-1439::DEBUG::2012-12-09 12:36:12,272::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1439::DEBUG::2012-12-09 12:36:12,273::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1440::DEBUG::2012-12-09 12:36:12,357::task::588::TaskManager.Task::(_updateState) Task=`04efecbf-fe4f-4f1d-8998-86659333c965`::moving from state init -> state preparing
>Thread-1440::INFO::2012-12-09 12:36:12,357::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1440::INFO::2012-12-09 12:36:12,358::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168499946594', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-1440::DEBUG::2012-12-09 12:36:12,358::task::1172::TaskManager.Task::(prepare) Task=`04efecbf-fe4f-4f1d-8998-86659333c965`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0168499946594', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.7', 'code': 358, 'valid': False}}
>Thread-1440::DEBUG::2012-12-09 12:36:12,358::task::588::TaskManager.Task::(_updateState) Task=`04efecbf-fe4f-4f1d-8998-86659333c965`::moving from state preparing -> state finished
>Thread-1440::DEBUG::2012-12-09 12:36:12,359::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1440::DEBUG::2012-12-09 12:36:12,359::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1440::DEBUG::2012-12-09 12:36:12,359::task::978::TaskManager.Task::(_decref) Task=`04efecbf-fe4f-4f1d-8998-86659333c965`::ref 0 aborting False
>Thread-1439::DEBUG::2012-12-09 12:36:12,378::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1439::WARNING::2012-12-09 12:36:12,379::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1439::DEBUG::2012-12-09 12:36:12,379::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1439::WARNING::2012-12-09 12:36:12,380::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1439::INFO::2012-12-09 12:36:12,380::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1439::DEBUG::2012-12-09 12:36:12,380::task::1172::TaskManager.Task::(prepare) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1439::DEBUG::2012-12-09 12:36:12,381::task::588::TaskManager.Task::(_updateState) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::moving from state preparing -> state finished
>Thread-1439::DEBUG::2012-12-09 12:36:12,381::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1439::DEBUG::2012-12-09 12:36:12,381::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1439::DEBUG::2012-12-09 12:36:12,381::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1439::DEBUG::2012-12-09 12:36:12,381::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1439::DEBUG::2012-12-09 12:36:12,382::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1439::DEBUG::2012-12-09 12:36:12,382::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1439::DEBUG::2012-12-09 12:36:12,382::task::978::TaskManager.Task::(_decref) Task=`a4a09a57-9c7d-4a7b-abb5-39f0682028ff`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:13,900::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:13,943::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193704 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:15,950::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:15,993::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189205 s, 54.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:18,000::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:18,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256496 s, 39.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:19,036::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:19,051::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000797785 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:20,056::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:20,107::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265939 s, 38.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:21,657::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:21,658::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:21,769::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:21,772::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:21,772::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:21,773::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:22,114::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:22,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275408 s, 37.2 MB/s\n'; <rc> = 0
>Thread-1446::DEBUG::2012-12-09 12:36:22,608::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1446::DEBUG::2012-12-09 12:36:22,609::task::588::TaskManager.Task::(_updateState) Task=`0d9cb43b-74a6-44b3-b24e-b8597025c06a`::moving from state init -> state preparing
>Thread-1446::INFO::2012-12-09 12:36:22,609::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1446::INFO::2012-12-09 12:36:22,610::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1446::DEBUG::2012-12-09 12:36:22,610::task::1172::TaskManager.Task::(prepare) Task=`0d9cb43b-74a6-44b3-b24e-b8597025c06a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1446::DEBUG::2012-12-09 12:36:22,611::task::588::TaskManager.Task::(_updateState) Task=`0d9cb43b-74a6-44b3-b24e-b8597025c06a`::moving from state preparing -> state finished
>Thread-1446::DEBUG::2012-12-09 12:36:22,611::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1446::DEBUG::2012-12-09 12:36:22,612::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1446::DEBUG::2012-12-09 12:36:22,612::task::978::TaskManager.Task::(_decref) Task=`0d9cb43b-74a6-44b3-b24e-b8597025c06a`::ref 0 aborting False
>Thread-1447::DEBUG::2012-12-09 12:36:22,630::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1447::DEBUG::2012-12-09 12:36:22,631::task::588::TaskManager.Task::(_updateState) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::moving from state init -> state preparing
>Thread-1447::INFO::2012-12-09 12:36:22,631::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1447::DEBUG::2012-12-09 12:36:22,632::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`15cc5d59-2521-42b0-9b4f-256caa949dc1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1447::DEBUG::2012-12-09 12:36:22,632::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1447::DEBUG::2012-12-09 12:36:22,633::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1447::DEBUG::2012-12-09 12:36:22,633::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`15cc5d59-2521-42b0-9b4f-256caa949dc1`::Granted request
>Thread-1447::DEBUG::2012-12-09 12:36:22,634::task::817::TaskManager.Task::(resourceAcquired) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1447::DEBUG::2012-12-09 12:36:22,634::task::978::TaskManager.Task::(_decref) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::ref 1 aborting False
>Thread-1447::DEBUG::2012-12-09 12:36:22,636::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1447::DEBUG::2012-12-09 12:36:22,637::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1448::DEBUG::2012-12-09 12:36:22,710::task::588::TaskManager.Task::(_updateState) Task=`801fb89f-316c-4b81-8e64-3832ba7e153f`::moving from state init -> state preparing
>Thread-1448::INFO::2012-12-09 12:36:22,711::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1448::INFO::2012-12-09 12:36:22,711::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162718296051', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-1448::DEBUG::2012-12-09 12:36:22,711::task::1172::TaskManager.Task::(prepare) Task=`801fb89f-316c-4b81-8e64-3832ba7e153f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162718296051', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-1448::DEBUG::2012-12-09 12:36:22,711::task::588::TaskManager.Task::(_updateState) Task=`801fb89f-316c-4b81-8e64-3832ba7e153f`::moving from state preparing -> state finished
>Thread-1448::DEBUG::2012-12-09 12:36:22,712::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1448::DEBUG::2012-12-09 12:36:22,712::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1448::DEBUG::2012-12-09 12:36:22,712::task::978::TaskManager.Task::(_decref) Task=`801fb89f-316c-4b81-8e64-3832ba7e153f`::ref 0 aborting False
>Thread-1447::DEBUG::2012-12-09 12:36:22,756::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1447::WARNING::2012-12-09 12:36:22,758::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1447::DEBUG::2012-12-09 12:36:22,758::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1447::WARNING::2012-12-09 12:36:22,759::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1447::INFO::2012-12-09 12:36:22,759::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1447::DEBUG::2012-12-09 12:36:22,759::task::1172::TaskManager.Task::(prepare) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1447::DEBUG::2012-12-09 12:36:22,760::task::588::TaskManager.Task::(_updateState) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::moving from state preparing -> state finished
>Thread-1447::DEBUG::2012-12-09 12:36:22,760::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1447::DEBUG::2012-12-09 12:36:22,760::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1447::DEBUG::2012-12-09 12:36:22,760::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1447::DEBUG::2012-12-09 12:36:22,761::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1447::DEBUG::2012-12-09 12:36:22,761::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1447::DEBUG::2012-12-09 12:36:22,761::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1447::DEBUG::2012-12-09 12:36:22,761::task::978::TaskManager.Task::(_decref) Task=`1e58d191-8f29-43b1-980f-1013d796468b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:24,172::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:24,215::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199689 s, 51.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:26,222::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:26,270::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023665 s, 43.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:28,276::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:28,327::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266358 s, 38.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:29,056::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:29,071::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000852549 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:30,334::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:30,381::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225423 s, 45.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:31,774::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:31,776::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:31,890::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:31,892::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:31,892::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:31,893::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:32,387::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:32,440::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256935 s, 39.9 MB/s\n'; <rc> = 0
>Thread-1454::DEBUG::2012-12-09 12:36:32,888::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1454::DEBUG::2012-12-09 12:36:32,889::task::588::TaskManager.Task::(_updateState) Task=`75401e05-5876-47dc-948d-0cb598bee4ed`::moving from state init -> state preparing
>Thread-1454::INFO::2012-12-09 12:36:32,889::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1454::INFO::2012-12-09 12:36:32,890::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1454::DEBUG::2012-12-09 12:36:32,890::task::1172::TaskManager.Task::(prepare) Task=`75401e05-5876-47dc-948d-0cb598bee4ed`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1454::DEBUG::2012-12-09 12:36:32,891::task::588::TaskManager.Task::(_updateState) Task=`75401e05-5876-47dc-948d-0cb598bee4ed`::moving from state preparing -> state finished
>Thread-1454::DEBUG::2012-12-09 12:36:32,891::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1454::DEBUG::2012-12-09 12:36:32,892::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1454::DEBUG::2012-12-09 12:36:32,892::task::978::TaskManager.Task::(_decref) Task=`75401e05-5876-47dc-948d-0cb598bee4ed`::ref 0 aborting False
>Thread-1455::DEBUG::2012-12-09 12:36:32,915::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1455::DEBUG::2012-12-09 12:36:32,916::task::588::TaskManager.Task::(_updateState) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::moving from state init -> state preparing
>Thread-1455::INFO::2012-12-09 12:36:32,916::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1455::DEBUG::2012-12-09 12:36:32,916::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df4e818d-1f5b-4618-bcb3-f740e0befea0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1455::DEBUG::2012-12-09 12:36:32,916::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1455::DEBUG::2012-12-09 12:36:32,917::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1455::DEBUG::2012-12-09 12:36:32,917::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df4e818d-1f5b-4618-bcb3-f740e0befea0`::Granted request
>Thread-1455::DEBUG::2012-12-09 12:36:32,917::task::817::TaskManager.Task::(resourceAcquired) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1455::DEBUG::2012-12-09 12:36:32,917::task::978::TaskManager.Task::(_decref) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::ref 1 aborting False
>Thread-1455::DEBUG::2012-12-09 12:36:32,918::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1455::DEBUG::2012-12-09 12:36:32,919::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1456::DEBUG::2012-12-09 12:36:32,980::task::588::TaskManager.Task::(_updateState) Task=`ee151255-a030-4fcd-aac4-67218265b3d5`::moving from state init -> state preparing
>Thread-1456::INFO::2012-12-09 12:36:32,981::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1456::INFO::2012-12-09 12:36:32,981::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167248249054', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-1456::DEBUG::2012-12-09 12:36:32,981::task::1172::TaskManager.Task::(prepare) Task=`ee151255-a030-4fcd-aac4-67218265b3d5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167248249054', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-1456::DEBUG::2012-12-09 12:36:32,981::task::588::TaskManager.Task::(_updateState) Task=`ee151255-a030-4fcd-aac4-67218265b3d5`::moving from state preparing -> state finished
>Thread-1456::DEBUG::2012-12-09 12:36:32,982::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1456::DEBUG::2012-12-09 12:36:32,982::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1456::DEBUG::2012-12-09 12:36:32,982::task::978::TaskManager.Task::(_decref) Task=`ee151255-a030-4fcd-aac4-67218265b3d5`::ref 0 aborting False
>Thread-1455::DEBUG::2012-12-09 12:36:33,028::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1455::WARNING::2012-12-09 12:36:33,030::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1455::DEBUG::2012-12-09 12:36:33,031::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1455::WARNING::2012-12-09 12:36:33,032::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1455::INFO::2012-12-09 12:36:33,033::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1455::DEBUG::2012-12-09 12:36:33,033::task::1172::TaskManager.Task::(prepare) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1455::DEBUG::2012-12-09 12:36:33,033::task::588::TaskManager.Task::(_updateState) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::moving from state preparing -> state finished
>Thread-1455::DEBUG::2012-12-09 12:36:33,034::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1455::DEBUG::2012-12-09 12:36:33,034::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1455::DEBUG::2012-12-09 12:36:33,034::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1455::DEBUG::2012-12-09 12:36:33,034::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1455::DEBUG::2012-12-09 12:36:33,034::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1455::DEBUG::2012-12-09 12:36:33,035::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1455::DEBUG::2012-12-09 12:36:33,035::task::978::TaskManager.Task::(_decref) Task=`8b7d0f3a-a156-4db1-9dbe-ba10a412d8e5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:34,446::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:34,497::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264087 s, 38.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:36,504::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:36,557::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279431 s, 36.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:38,561::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:38,610::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239461 s, 42.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:39,076::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:39,092::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000912571 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:40,616::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:40,667::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261265 s, 39.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:41,894::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:41,896::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:42,001::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:42,002::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:42,002::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:42,003::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:42,673::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:42,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236298 s, 43.3 MB/s\n'; <rc> = 0
>Thread-1462::DEBUG::2012-12-09 12:36:43,155::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1462::DEBUG::2012-12-09 12:36:43,156::task::588::TaskManager.Task::(_updateState) Task=`e43e340d-f218-4c32-9ba4-f8efd362bed3`::moving from state init -> state preparing
>Thread-1462::INFO::2012-12-09 12:36:43,157::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1462::INFO::2012-12-09 12:36:43,157::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1462::DEBUG::2012-12-09 12:36:43,158::task::1172::TaskManager.Task::(prepare) Task=`e43e340d-f218-4c32-9ba4-f8efd362bed3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1462::DEBUG::2012-12-09 12:36:43,158::task::588::TaskManager.Task::(_updateState) Task=`e43e340d-f218-4c32-9ba4-f8efd362bed3`::moving from state preparing -> state finished
>Thread-1462::DEBUG::2012-12-09 12:36:43,158::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1462::DEBUG::2012-12-09 12:36:43,159::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1462::DEBUG::2012-12-09 12:36:43,159::task::978::TaskManager.Task::(_decref) Task=`e43e340d-f218-4c32-9ba4-f8efd362bed3`::ref 0 aborting False
>Thread-1463::DEBUG::2012-12-09 12:36:43,179::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1463::DEBUG::2012-12-09 12:36:43,179::task::588::TaskManager.Task::(_updateState) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::moving from state init -> state preparing
>Thread-1463::INFO::2012-12-09 12:36:43,179::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1463::DEBUG::2012-12-09 12:36:43,180::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e89e345a-3bfa-4b09-acb7-4c675e92119f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1463::DEBUG::2012-12-09 12:36:43,180::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1463::DEBUG::2012-12-09 12:36:43,180::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1463::DEBUG::2012-12-09 12:36:43,180::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e89e345a-3bfa-4b09-acb7-4c675e92119f`::Granted request
>Thread-1463::DEBUG::2012-12-09 12:36:43,181::task::817::TaskManager.Task::(resourceAcquired) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1463::DEBUG::2012-12-09 12:36:43,181::task::978::TaskManager.Task::(_decref) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::ref 1 aborting False
>Thread-1463::DEBUG::2012-12-09 12:36:43,181::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1463::DEBUG::2012-12-09 12:36:43,182::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1464::DEBUG::2012-12-09 12:36:43,251::task::588::TaskManager.Task::(_updateState) Task=`ab2b713c-4cec-4d7e-a72f-6434e1d4a50e`::moving from state init -> state preparing
>Thread-1464::INFO::2012-12-09 12:36:43,251::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1464::INFO::2012-12-09 12:36:43,251::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0173840522766', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-1464::DEBUG::2012-12-09 12:36:43,252::task::1172::TaskManager.Task::(prepare) Task=`ab2b713c-4cec-4d7e-a72f-6434e1d4a50e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0173840522766', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-1464::DEBUG::2012-12-09 12:36:43,252::task::588::TaskManager.Task::(_updateState) Task=`ab2b713c-4cec-4d7e-a72f-6434e1d4a50e`::moving from state preparing -> state finished
>Thread-1464::DEBUG::2012-12-09 12:36:43,252::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1464::DEBUG::2012-12-09 12:36:43,252::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1464::DEBUG::2012-12-09 12:36:43,252::task::978::TaskManager.Task::(_decref) Task=`ab2b713c-4cec-4d7e-a72f-6434e1d4a50e`::ref 0 aborting False
>Thread-1463::DEBUG::2012-12-09 12:36:43,292::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1463::WARNING::2012-12-09 12:36:43,294::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1463::DEBUG::2012-12-09 12:36:43,294::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1463::WARNING::2012-12-09 12:36:43,294::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1463::INFO::2012-12-09 12:36:43,295::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1463::DEBUG::2012-12-09 12:36:43,295::task::1172::TaskManager.Task::(prepare) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1463::DEBUG::2012-12-09 12:36:43,295::task::588::TaskManager.Task::(_updateState) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::moving from state preparing -> state finished
>Thread-1463::DEBUG::2012-12-09 12:36:43,295::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1463::DEBUG::2012-12-09 12:36:43,296::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1463::DEBUG::2012-12-09 12:36:43,296::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1463::DEBUG::2012-12-09 12:36:43,296::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1463::DEBUG::2012-12-09 12:36:43,296::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1463::DEBUG::2012-12-09 12:36:43,296::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1463::DEBUG::2012-12-09 12:36:43,297::task::978::TaskManager.Task::(_decref) Task=`3a3507d5-2c10-43c2-b5b3-21d08417bc14`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:44,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:44,779::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239726 s, 42.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:46,785::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:46,837::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281165 s, 36.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:48,844::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:48,896::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277252 s, 36.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:49,098::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:49,111::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000923199 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:50,902::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:50,944::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192773 s, 53.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:36:52,004::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:36:52,005::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:36:52,111::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:36:52,114::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:36:52,114::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:36:52,114::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:36:52,951::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:52,999::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023336 s, 43.9 MB/s\n'; <rc> = 0
>Thread-1470::DEBUG::2012-12-09 12:36:53,379::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1470::DEBUG::2012-12-09 12:36:53,380::task::588::TaskManager.Task::(_updateState) Task=`80b2800f-72cc-45b5-913c-013496c6ba57`::moving from state init -> state preparing
>Thread-1470::INFO::2012-12-09 12:36:53,381::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1470::INFO::2012-12-09 12:36:53,381::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1470::DEBUG::2012-12-09 12:36:53,382::task::1172::TaskManager.Task::(prepare) Task=`80b2800f-72cc-45b5-913c-013496c6ba57`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1470::DEBUG::2012-12-09 12:36:53,382::task::588::TaskManager.Task::(_updateState) Task=`80b2800f-72cc-45b5-913c-013496c6ba57`::moving from state preparing -> state finished
>Thread-1470::DEBUG::2012-12-09 12:36:53,383::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1470::DEBUG::2012-12-09 12:36:53,383::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1470::DEBUG::2012-12-09 12:36:53,383::task::978::TaskManager.Task::(_decref) Task=`80b2800f-72cc-45b5-913c-013496c6ba57`::ref 0 aborting False
>Thread-1471::DEBUG::2012-12-09 12:36:53,398::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1471::DEBUG::2012-12-09 12:36:53,399::task::588::TaskManager.Task::(_updateState) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::moving from state init -> state preparing
>Thread-1471::INFO::2012-12-09 12:36:53,399::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1471::DEBUG::2012-12-09 12:36:53,400::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8de0b94d-75a7-4bfe-9536-a16faac02195`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1471::DEBUG::2012-12-09 12:36:53,400::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1471::DEBUG::2012-12-09 12:36:53,401::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1471::DEBUG::2012-12-09 12:36:53,401::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8de0b94d-75a7-4bfe-9536-a16faac02195`::Granted request
>Thread-1471::DEBUG::2012-12-09 12:36:53,402::task::817::TaskManager.Task::(resourceAcquired) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1471::DEBUG::2012-12-09 12:36:53,402::task::978::TaskManager.Task::(_decref) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::ref 1 aborting False
>Thread-1471::DEBUG::2012-12-09 12:36:53,404::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1471::DEBUG::2012-12-09 12:36:53,404::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1472::DEBUG::2012-12-09 12:36:53,492::task::588::TaskManager.Task::(_updateState) Task=`15fa08bf-f7e6-4eb5-9587-92ca9cbdbf14`::moving from state init -> state preparing
>Thread-1472::INFO::2012-12-09 12:36:53,492::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1472::INFO::2012-12-09 12:36:53,493::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138919353485', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-1472::DEBUG::2012-12-09 12:36:53,493::task::1172::TaskManager.Task::(prepare) Task=`15fa08bf-f7e6-4eb5-9587-92ca9cbdbf14`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138919353485', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-1472::DEBUG::2012-12-09 12:36:53,493::task::588::TaskManager.Task::(_updateState) Task=`15fa08bf-f7e6-4eb5-9587-92ca9cbdbf14`::moving from state preparing -> state finished
>Thread-1472::DEBUG::2012-12-09 12:36:53,493::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1472::DEBUG::2012-12-09 12:36:53,493::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1472::DEBUG::2012-12-09 12:36:53,494::task::978::TaskManager.Task::(_decref) Task=`15fa08bf-f7e6-4eb5-9587-92ca9cbdbf14`::ref 0 aborting False
>Thread-1471::DEBUG::2012-12-09 12:36:53,505::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1471::WARNING::2012-12-09 12:36:53,506::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1471::DEBUG::2012-12-09 12:36:53,506::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1471::WARNING::2012-12-09 12:36:53,507::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1471::INFO::2012-12-09 12:36:53,507::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1471::DEBUG::2012-12-09 12:36:53,507::task::1172::TaskManager.Task::(prepare) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1471::DEBUG::2012-12-09 12:36:53,507::task::588::TaskManager.Task::(_updateState) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::moving from state preparing -> state finished
>Thread-1471::DEBUG::2012-12-09 12:36:53,508::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1471::DEBUG::2012-12-09 12:36:53,508::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1471::DEBUG::2012-12-09 12:36:53,508::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1471::DEBUG::2012-12-09 12:36:53,508::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1471::DEBUG::2012-12-09 12:36:53,508::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1471::DEBUG::2012-12-09 12:36:53,509::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1471::DEBUG::2012-12-09 12:36:53,509::task::978::TaskManager.Task::(_decref) Task=`a7ecd618-c667-4d83-826c-892d3fc458a6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:36:55,005::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:55,057::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270159 s, 37.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:57,063::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:57,110::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225331 s, 45.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:36:59,116::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:36:59,123::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:36:59,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00219437 s, 1.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:36:59,188::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372101 s, 27.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:01,195::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:01,247::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271128 s, 37.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:02,115::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:02,117::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:02,232::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:02,233::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:02,234::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:02,234::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:03,254::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:03,314::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0339473 s, 30.2 MB/s\n'; <rc> = 0
>Thread-1478::DEBUG::2012-12-09 12:37:03,808::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1478::DEBUG::2012-12-09 12:37:03,809::task::588::TaskManager.Task::(_updateState) Task=`da066455-b81f-42ab-abf3-ef46d98b6e51`::moving from state init -> state preparing
>Thread-1478::INFO::2012-12-09 12:37:03,810::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1478::INFO::2012-12-09 12:37:03,811::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1478::DEBUG::2012-12-09 12:37:03,811::task::1172::TaskManager.Task::(prepare) Task=`da066455-b81f-42ab-abf3-ef46d98b6e51`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1478::DEBUG::2012-12-09 12:37:03,812::task::588::TaskManager.Task::(_updateState) Task=`da066455-b81f-42ab-abf3-ef46d98b6e51`::moving from state preparing -> state finished
>Thread-1478::DEBUG::2012-12-09 12:37:03,812::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1478::DEBUG::2012-12-09 12:37:03,812::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1478::DEBUG::2012-12-09 12:37:03,813::task::978::TaskManager.Task::(_decref) Task=`da066455-b81f-42ab-abf3-ef46d98b6e51`::ref 0 aborting False
>Thread-1479::DEBUG::2012-12-09 12:37:03,829::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1479::DEBUG::2012-12-09 12:37:03,829::task::588::TaskManager.Task::(_updateState) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::moving from state init -> state preparing
>Thread-1479::INFO::2012-12-09 12:37:03,830::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1479::DEBUG::2012-12-09 12:37:03,831::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3a513846-2a07-4120-adf2-469e75bc591a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1479::DEBUG::2012-12-09 12:37:03,831::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1479::DEBUG::2012-12-09 12:37:03,832::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1479::DEBUG::2012-12-09 12:37:03,832::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3a513846-2a07-4120-adf2-469e75bc591a`::Granted request
>Thread-1479::DEBUG::2012-12-09 12:37:03,833::task::817::TaskManager.Task::(resourceAcquired) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1479::DEBUG::2012-12-09 12:37:03,833::task::978::TaskManager.Task::(_decref) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::ref 1 aborting False
>Thread-1479::DEBUG::2012-12-09 12:37:03,835::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1479::DEBUG::2012-12-09 12:37:03,836::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1480::DEBUG::2012-12-09 12:37:03,926::task::588::TaskManager.Task::(_updateState) Task=`68a38ea4-2742-49c9-b346-d0db7cf6907a`::moving from state init -> state preparing
>Thread-1480::INFO::2012-12-09 12:37:03,926::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1480::INFO::2012-12-09 12:37:03,927::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0226480960846', 'lastCheck': '4.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-1480::DEBUG::2012-12-09 12:37:03,927::task::1172::TaskManager.Task::(prepare) Task=`68a38ea4-2742-49c9-b346-d0db7cf6907a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0226480960846', 'lastCheck': '4.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-1480::DEBUG::2012-12-09 12:37:03,927::task::588::TaskManager.Task::(_updateState) Task=`68a38ea4-2742-49c9-b346-d0db7cf6907a`::moving from state preparing -> state finished
>Thread-1480::DEBUG::2012-12-09 12:37:03,927::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1480::DEBUG::2012-12-09 12:37:03,927::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1480::DEBUG::2012-12-09 12:37:03,928::task::978::TaskManager.Task::(_decref) Task=`68a38ea4-2742-49c9-b346-d0db7cf6907a`::ref 0 aborting False
>Thread-1479::DEBUG::2012-12-09 12:37:03,943::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1479::WARNING::2012-12-09 12:37:03,944::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1479::DEBUG::2012-12-09 12:37:03,944::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1479::WARNING::2012-12-09 12:37:03,945::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1479::INFO::2012-12-09 12:37:03,945::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1479::DEBUG::2012-12-09 12:37:03,945::task::1172::TaskManager.Task::(prepare) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1479::DEBUG::2012-12-09 12:37:03,946::task::588::TaskManager.Task::(_updateState) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::moving from state preparing -> state finished
>Thread-1479::DEBUG::2012-12-09 12:37:03,946::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1479::DEBUG::2012-12-09 12:37:03,946::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1479::DEBUG::2012-12-09 12:37:03,946::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1479::DEBUG::2012-12-09 12:37:03,947::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1479::DEBUG::2012-12-09 12:37:03,947::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1479::DEBUG::2012-12-09 12:37:03,947::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1479::DEBUG::2012-12-09 12:37:03,947::task::978::TaskManager.Task::(_decref) Task=`e4b82b70-f1c4-4236-8825-cde4864bbf28`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:05,320::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:05,368::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240592 s, 42.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:07,375::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:07,427::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272848 s, 37.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:09,152::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:09,165::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000927717 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:09,434::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:09,480::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226712 s, 45.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:11,487::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:11,532::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01959 s, 52.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:12,235::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:12,237::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:12,348::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:12,350::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:12,351::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:12,352::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:13,538::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:13,587::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024991 s, 41.0 MB/s\n'; <rc> = 0
>Thread-1486::DEBUG::2012-12-09 12:37:14,231::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1486::DEBUG::2012-12-09 12:37:14,232::task::588::TaskManager.Task::(_updateState) Task=`e628d3a0-b0dc-4a5e-b253-4fc10606c933`::moving from state init -> state preparing
>Thread-1486::INFO::2012-12-09 12:37:14,233::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1486::INFO::2012-12-09 12:37:14,233::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1486::DEBUG::2012-12-09 12:37:14,234::task::1172::TaskManager.Task::(prepare) Task=`e628d3a0-b0dc-4a5e-b253-4fc10606c933`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1486::DEBUG::2012-12-09 12:37:14,234::task::588::TaskManager.Task::(_updateState) Task=`e628d3a0-b0dc-4a5e-b253-4fc10606c933`::moving from state preparing -> state finished
>Thread-1486::DEBUG::2012-12-09 12:37:14,235::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1486::DEBUG::2012-12-09 12:37:14,235::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1486::DEBUG::2012-12-09 12:37:14,235::task::978::TaskManager.Task::(_decref) Task=`e628d3a0-b0dc-4a5e-b253-4fc10606c933`::ref 0 aborting False
>Thread-1487::DEBUG::2012-12-09 12:37:14,257::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1487::DEBUG::2012-12-09 12:37:14,258::task::588::TaskManager.Task::(_updateState) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::moving from state init -> state preparing
>Thread-1487::INFO::2012-12-09 12:37:14,258::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1487::DEBUG::2012-12-09 12:37:14,259::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e9d5809d-98e1-4b5f-b101-0f7c7f9db1a5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1487::DEBUG::2012-12-09 12:37:14,260::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1487::DEBUG::2012-12-09 12:37:14,260::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1487::DEBUG::2012-12-09 12:37:14,261::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e9d5809d-98e1-4b5f-b101-0f7c7f9db1a5`::Granted request
>Thread-1487::DEBUG::2012-12-09 12:37:14,261::task::817::TaskManager.Task::(resourceAcquired) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1487::DEBUG::2012-12-09 12:37:14,262::task::978::TaskManager.Task::(_decref) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::ref 1 aborting False
>Thread-1487::DEBUG::2012-12-09 12:37:14,263::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1487::DEBUG::2012-12-09 12:37:14,264::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1487::DEBUG::2012-12-09 12:37:14,376::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1487::WARNING::2012-12-09 12:37:14,377::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1487::DEBUG::2012-12-09 12:37:14,377::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1487::WARNING::2012-12-09 12:37:14,378::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1487::INFO::2012-12-09 12:37:14,378::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1487::DEBUG::2012-12-09 12:37:14,379::task::1172::TaskManager.Task::(prepare) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1487::DEBUG::2012-12-09 12:37:14,380::task::588::TaskManager.Task::(_updateState) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::moving from state preparing -> state finished
>Thread-1487::DEBUG::2012-12-09 12:37:14,380::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1487::DEBUG::2012-12-09 12:37:14,380::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1487::DEBUG::2012-12-09 12:37:14,381::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1487::DEBUG::2012-12-09 12:37:14,381::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1487::DEBUG::2012-12-09 12:37:14,381::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1487::DEBUG::2012-12-09 12:37:14,382::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1487::DEBUG::2012-12-09 12:37:14,382::task::978::TaskManager.Task::(_decref) Task=`2fe3c03d-e3a2-43f5-a30c-8e1e8e6656cf`::ref 0 aborting False
>Thread-1488::DEBUG::2012-12-09 12:37:14,390::task::588::TaskManager.Task::(_updateState) Task=`848ed534-e58f-4a5a-a580-097bc9d3cc63`::moving from state init -> state preparing
>Thread-1488::INFO::2012-12-09 12:37:14,391::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1488::INFO::2012-12-09 12:37:14,392::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137209892273', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-1488::DEBUG::2012-12-09 12:37:14,392::task::1172::TaskManager.Task::(prepare) Task=`848ed534-e58f-4a5a-a580-097bc9d3cc63`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137209892273', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-1488::DEBUG::2012-12-09 12:37:14,393::task::588::TaskManager.Task::(_updateState) Task=`848ed534-e58f-4a5a-a580-097bc9d3cc63`::moving from state preparing -> state finished
>Thread-1488::DEBUG::2012-12-09 12:37:14,393::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1488::DEBUG::2012-12-09 12:37:14,394::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1488::DEBUG::2012-12-09 12:37:14,394::task::978::TaskManager.Task::(_decref) Task=`848ed534-e58f-4a5a-a580-097bc9d3cc63`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:15,594::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:15,633::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016554 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:17,640::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:17,688::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240779 s, 42.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:19,169::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:19,186::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000803048 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:19,695::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:19,741::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218956 s, 46.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:21,748::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:21,800::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266167 s, 38.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:22,353::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:22,354::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:22,472::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:22,474::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:22,474::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:22,475::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:23,806::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:23,845::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166088 s, 61.7 MB/s\n'; <rc> = 0
>Thread-1494::DEBUG::2012-12-09 12:37:24,458::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1494::DEBUG::2012-12-09 12:37:24,459::task::588::TaskManager.Task::(_updateState) Task=`8e0d5ca1-992c-43ee-b96c-d1922992d0c1`::moving from state init -> state preparing
>Thread-1494::INFO::2012-12-09 12:37:24,459::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1494::INFO::2012-12-09 12:37:24,460::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1494::DEBUG::2012-12-09 12:37:24,460::task::1172::TaskManager.Task::(prepare) Task=`8e0d5ca1-992c-43ee-b96c-d1922992d0c1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1494::DEBUG::2012-12-09 12:37:24,461::task::588::TaskManager.Task::(_updateState) Task=`8e0d5ca1-992c-43ee-b96c-d1922992d0c1`::moving from state preparing -> state finished
>Thread-1494::DEBUG::2012-12-09 12:37:24,461::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1494::DEBUG::2012-12-09 12:37:24,461::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1494::DEBUG::2012-12-09 12:37:24,462::task::978::TaskManager.Task::(_decref) Task=`8e0d5ca1-992c-43ee-b96c-d1922992d0c1`::ref 0 aborting False
>Thread-1495::DEBUG::2012-12-09 12:37:24,479::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1495::DEBUG::2012-12-09 12:37:24,479::task::588::TaskManager.Task::(_updateState) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::moving from state init -> state preparing
>Thread-1495::INFO::2012-12-09 12:37:24,480::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1495::DEBUG::2012-12-09 12:37:24,480::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d25a4752-3092-4087-8db3-cfc80245c176`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1495::DEBUG::2012-12-09 12:37:24,480::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1495::DEBUG::2012-12-09 12:37:24,480::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1495::DEBUG::2012-12-09 12:37:24,480::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d25a4752-3092-4087-8db3-cfc80245c176`::Granted request
>Thread-1495::DEBUG::2012-12-09 12:37:24,481::task::817::TaskManager.Task::(resourceAcquired) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1495::DEBUG::2012-12-09 12:37:24,481::task::978::TaskManager.Task::(_decref) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::ref 1 aborting False
>Thread-1495::DEBUG::2012-12-09 12:37:24,482::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1495::DEBUG::2012-12-09 12:37:24,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1495::DEBUG::2012-12-09 12:37:24,585::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1495::WARNING::2012-12-09 12:37:24,587::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1495::DEBUG::2012-12-09 12:37:24,588::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1495::WARNING::2012-12-09 12:37:24,589::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1495::INFO::2012-12-09 12:37:24,590::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1495::DEBUG::2012-12-09 12:37:24,590::task::1172::TaskManager.Task::(prepare) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1495::DEBUG::2012-12-09 12:37:24,590::task::588::TaskManager.Task::(_updateState) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::moving from state preparing -> state finished
>Thread-1495::DEBUG::2012-12-09 12:37:24,591::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1495::DEBUG::2012-12-09 12:37:24,591::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1495::DEBUG::2012-12-09 12:37:24,592::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1495::DEBUG::2012-12-09 12:37:24,592::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1495::DEBUG::2012-12-09 12:37:24,593::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1495::DEBUG::2012-12-09 12:37:24,593::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1495::DEBUG::2012-12-09 12:37:24,594::task::978::TaskManager.Task::(_decref) Task=`766ff812-2c36-4ba7-aaaa-d4dc995fb6a0`::ref 0 aborting False
>Thread-1496::DEBUG::2012-12-09 12:37:24,694::task::588::TaskManager.Task::(_updateState) Task=`dd8d6e84-fcf0-4f06-99c5-e41ca3392dfd`::moving from state init -> state preparing
>Thread-1496::INFO::2012-12-09 12:37:24,695::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1496::INFO::2012-12-09 12:37:24,695::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0182001590729', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-1496::DEBUG::2012-12-09 12:37:24,696::task::1172::TaskManager.Task::(prepare) Task=`dd8d6e84-fcf0-4f06-99c5-e41ca3392dfd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0182001590729', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-1496::DEBUG::2012-12-09 12:37:24,697::task::588::TaskManager.Task::(_updateState) Task=`dd8d6e84-fcf0-4f06-99c5-e41ca3392dfd`::moving from state preparing -> state finished
>Thread-1496::DEBUG::2012-12-09 12:37:24,697::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1496::DEBUG::2012-12-09 12:37:24,697::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1496::DEBUG::2012-12-09 12:37:24,698::task::978::TaskManager.Task::(_decref) Task=`dd8d6e84-fcf0-4f06-99c5-e41ca3392dfd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:25,852::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:25,893::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186001 s, 55.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:27,900::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:27,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025399 s, 40.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:29,192::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:29,201::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000684909 s, 6.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:29,956::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:30,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275307 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:32,014::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:32,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218938 s, 46.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:32,476::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:32,477::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:32,593::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:32,595::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:32,595::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:32,595::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:34,067::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:34,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257424 s, 39.8 MB/s\n'; <rc> = 0
>Thread-1502::DEBUG::2012-12-09 12:37:34,818::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1502::DEBUG::2012-12-09 12:37:34,819::task::588::TaskManager.Task::(_updateState) Task=`246516ac-3a4c-41cb-8d25-64314e1a2a20`::moving from state init -> state preparing
>Thread-1502::INFO::2012-12-09 12:37:34,819::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1502::INFO::2012-12-09 12:37:34,820::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1502::DEBUG::2012-12-09 12:37:34,820::task::1172::TaskManager.Task::(prepare) Task=`246516ac-3a4c-41cb-8d25-64314e1a2a20`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1502::DEBUG::2012-12-09 12:37:34,821::task::588::TaskManager.Task::(_updateState) Task=`246516ac-3a4c-41cb-8d25-64314e1a2a20`::moving from state preparing -> state finished
>Thread-1502::DEBUG::2012-12-09 12:37:34,821::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1502::DEBUG::2012-12-09 12:37:34,821::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1502::DEBUG::2012-12-09 12:37:34,821::task::978::TaskManager.Task::(_decref) Task=`246516ac-3a4c-41cb-8d25-64314e1a2a20`::ref 0 aborting False
>Thread-1503::DEBUG::2012-12-09 12:37:34,834::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1503::DEBUG::2012-12-09 12:37:34,835::task::588::TaskManager.Task::(_updateState) Task=`618a40c9-c104-47bc-9346-163081621cbd`::moving from state init -> state preparing
>Thread-1503::INFO::2012-12-09 12:37:34,836::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1503::DEBUG::2012-12-09 12:37:34,836::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6a64821-1129-414a-9303-f6ae03c7c45e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1503::DEBUG::2012-12-09 12:37:34,837::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1503::DEBUG::2012-12-09 12:37:34,837::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1503::DEBUG::2012-12-09 12:37:34,838::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b6a64821-1129-414a-9303-f6ae03c7c45e`::Granted request
>Thread-1503::DEBUG::2012-12-09 12:37:34,839::task::817::TaskManager.Task::(resourceAcquired) Task=`618a40c9-c104-47bc-9346-163081621cbd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1503::DEBUG::2012-12-09 12:37:34,839::task::978::TaskManager.Task::(_decref) Task=`618a40c9-c104-47bc-9346-163081621cbd`::ref 1 aborting False
>Thread-1503::DEBUG::2012-12-09 12:37:34,841::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1503::DEBUG::2012-12-09 12:37:34,841::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1503::DEBUG::2012-12-09 12:37:34,955::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1503::WARNING::2012-12-09 12:37:34,956::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1503::DEBUG::2012-12-09 12:37:34,957::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1503::WARNING::2012-12-09 12:37:34,957::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1503::INFO::2012-12-09 12:37:34,957::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1503::DEBUG::2012-12-09 12:37:34,958::task::1172::TaskManager.Task::(prepare) Task=`618a40c9-c104-47bc-9346-163081621cbd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1503::DEBUG::2012-12-09 12:37:34,958::task::588::TaskManager.Task::(_updateState) Task=`618a40c9-c104-47bc-9346-163081621cbd`::moving from state preparing -> state finished
>Thread-1503::DEBUG::2012-12-09 12:37:34,958::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1503::DEBUG::2012-12-09 12:37:34,958::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1503::DEBUG::2012-12-09 12:37:34,958::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1503::DEBUG::2012-12-09 12:37:34,959::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1503::DEBUG::2012-12-09 12:37:34,959::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1503::DEBUG::2012-12-09 12:37:34,959::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1503::DEBUG::2012-12-09 12:37:34,959::task::978::TaskManager.Task::(_decref) Task=`618a40c9-c104-47bc-9346-163081621cbd`::ref 0 aborting False
>Thread-1504::DEBUG::2012-12-09 12:37:35,017::task::588::TaskManager.Task::(_updateState) Task=`968e85f3-47d4-4d94-bb67-69c0eced5380`::moving from state init -> state preparing
>Thread-1504::INFO::2012-12-09 12:37:35,018::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1504::INFO::2012-12-09 12:37:35,019::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0094268321991', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-1504::DEBUG::2012-12-09 12:37:35,019::task::1172::TaskManager.Task::(prepare) Task=`968e85f3-47d4-4d94-bb67-69c0eced5380`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0094268321991', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-1504::DEBUG::2012-12-09 12:37:35,020::task::588::TaskManager.Task::(_updateState) Task=`968e85f3-47d4-4d94-bb67-69c0eced5380`::moving from state preparing -> state finished
>Thread-1504::DEBUG::2012-12-09 12:37:35,020::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1504::DEBUG::2012-12-09 12:37:35,021::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1504::DEBUG::2012-12-09 12:37:35,021::task::978::TaskManager.Task::(_decref) Task=`968e85f3-47d4-4d94-bb67-69c0eced5380`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:36,124::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:36,170::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214879 s, 47.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:38,177::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:38,218::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016721 s, 61.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:39,204::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:39,218::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000798156 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:40,225::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:40,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280769 s, 36.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:42,284::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:42,332::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234911 s, 43.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:42,596::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:42,597::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:42,691::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:42,692::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:42,692::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:42,693::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:44,339::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:44,378::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162126 s, 63.2 MB/s\n'; <rc> = 0
>Thread-1510::DEBUG::2012-12-09 12:37:45,086::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1510::DEBUG::2012-12-09 12:37:45,087::task::588::TaskManager.Task::(_updateState) Task=`78a7863b-3df2-461b-9981-25b08376a425`::moving from state init -> state preparing
>Thread-1510::INFO::2012-12-09 12:37:45,087::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1510::INFO::2012-12-09 12:37:45,087::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1510::DEBUG::2012-12-09 12:37:45,088::task::1172::TaskManager.Task::(prepare) Task=`78a7863b-3df2-461b-9981-25b08376a425`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1510::DEBUG::2012-12-09 12:37:45,088::task::588::TaskManager.Task::(_updateState) Task=`78a7863b-3df2-461b-9981-25b08376a425`::moving from state preparing -> state finished
>Thread-1510::DEBUG::2012-12-09 12:37:45,088::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1510::DEBUG::2012-12-09 12:37:45,088::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1510::DEBUG::2012-12-09 12:37:45,088::task::978::TaskManager.Task::(_decref) Task=`78a7863b-3df2-461b-9981-25b08376a425`::ref 0 aborting False
>Thread-1511::DEBUG::2012-12-09 12:37:45,100::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1511::DEBUG::2012-12-09 12:37:45,100::task::588::TaskManager.Task::(_updateState) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::moving from state init -> state preparing
>Thread-1511::INFO::2012-12-09 12:37:45,101::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1511::DEBUG::2012-12-09 12:37:45,101::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f76a518-72ab-496c-8019-7fef0114f011`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1511::DEBUG::2012-12-09 12:37:45,101::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1511::DEBUG::2012-12-09 12:37:45,101::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1511::DEBUG::2012-12-09 12:37:45,102::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f76a518-72ab-496c-8019-7fef0114f011`::Granted request
>Thread-1511::DEBUG::2012-12-09 12:37:45,102::task::817::TaskManager.Task::(resourceAcquired) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1511::DEBUG::2012-12-09 12:37:45,102::task::978::TaskManager.Task::(_decref) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::ref 1 aborting False
>Thread-1511::DEBUG::2012-12-09 12:37:45,103::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1511::DEBUG::2012-12-09 12:37:45,103::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1511::DEBUG::2012-12-09 12:37:45,198::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1511::WARNING::2012-12-09 12:37:45,199::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1511::DEBUG::2012-12-09 12:37:45,199::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1511::WARNING::2012-12-09 12:37:45,200::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1511::INFO::2012-12-09 12:37:45,200::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1511::DEBUG::2012-12-09 12:37:45,200::task::1172::TaskManager.Task::(prepare) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1511::DEBUG::2012-12-09 12:37:45,201::task::588::TaskManager.Task::(_updateState) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::moving from state preparing -> state finished
>Thread-1511::DEBUG::2012-12-09 12:37:45,201::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1511::DEBUG::2012-12-09 12:37:45,201::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1511::DEBUG::2012-12-09 12:37:45,201::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1511::DEBUG::2012-12-09 12:37:45,202::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1511::DEBUG::2012-12-09 12:37:45,202::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1511::DEBUG::2012-12-09 12:37:45,202::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1511::DEBUG::2012-12-09 12:37:45,202::task::978::TaskManager.Task::(_decref) Task=`2c9521e0-2b76-4330-8c71-f0e00aeb77aa`::ref 0 aborting False
>Thread-1512::DEBUG::2012-12-09 12:37:45,270::task::588::TaskManager.Task::(_updateState) Task=`0a944c30-c1a7-4f57-9410-9f3cf9baa72c`::moving from state init -> state preparing
>Thread-1512::INFO::2012-12-09 12:37:45,271::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1512::INFO::2012-12-09 12:37:45,272::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149939060211', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-1512::DEBUG::2012-12-09 12:37:45,273::task::1172::TaskManager.Task::(prepare) Task=`0a944c30-c1a7-4f57-9410-9f3cf9baa72c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149939060211', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-1512::DEBUG::2012-12-09 12:37:45,273::task::588::TaskManager.Task::(_updateState) Task=`0a944c30-c1a7-4f57-9410-9f3cf9baa72c`::moving from state preparing -> state finished
>Thread-1512::DEBUG::2012-12-09 12:37:45,274::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1512::DEBUG::2012-12-09 12:37:45,274::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1512::DEBUG::2012-12-09 12:37:45,275::task::978::TaskManager.Task::(_decref) Task=`0a944c30-c1a7-4f57-9410-9f3cf9baa72c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:46,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:46,422::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163709 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:48,429::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:48,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164189 s, 62.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:49,224::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:49,239::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000865081 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:50,474::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:50,513::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160335 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:52,519::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:52,559::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172194 s, 59.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:37:52,694::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:37:52,695::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:37:52,798::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:37:52,799::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:37:52,799::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:37:52,800::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:37:54,566::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:54,605::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162739 s, 62.9 MB/s\n'; <rc> = 0
>Thread-1518::DEBUG::2012-12-09 12:37:55,401::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1518::DEBUG::2012-12-09 12:37:55,402::task::588::TaskManager.Task::(_updateState) Task=`b47b698a-09e0-4f7a-b5ed-a4c49c8c4236`::moving from state init -> state preparing
>Thread-1518::INFO::2012-12-09 12:37:55,403::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1518::INFO::2012-12-09 12:37:55,403::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1518::DEBUG::2012-12-09 12:37:55,404::task::1172::TaskManager.Task::(prepare) Task=`b47b698a-09e0-4f7a-b5ed-a4c49c8c4236`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1518::DEBUG::2012-12-09 12:37:55,404::task::588::TaskManager.Task::(_updateState) Task=`b47b698a-09e0-4f7a-b5ed-a4c49c8c4236`::moving from state preparing -> state finished
>Thread-1518::DEBUG::2012-12-09 12:37:55,404::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1518::DEBUG::2012-12-09 12:37:55,405::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1518::DEBUG::2012-12-09 12:37:55,405::task::978::TaskManager.Task::(_decref) Task=`b47b698a-09e0-4f7a-b5ed-a4c49c8c4236`::ref 0 aborting False
>Thread-1519::DEBUG::2012-12-09 12:37:55,418::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1519::DEBUG::2012-12-09 12:37:55,419::task::588::TaskManager.Task::(_updateState) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::moving from state init -> state preparing
>Thread-1519::INFO::2012-12-09 12:37:55,420::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1519::DEBUG::2012-12-09 12:37:55,420::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df456241-c7f5-4126-9314-ee56ab9fb578`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1519::DEBUG::2012-12-09 12:37:55,421::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1519::DEBUG::2012-12-09 12:37:55,421::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1519::DEBUG::2012-12-09 12:37:55,421::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df456241-c7f5-4126-9314-ee56ab9fb578`::Granted request
>Thread-1519::DEBUG::2012-12-09 12:37:55,422::task::817::TaskManager.Task::(resourceAcquired) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1519::DEBUG::2012-12-09 12:37:55,422::task::978::TaskManager.Task::(_decref) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::ref 1 aborting False
>Thread-1519::DEBUG::2012-12-09 12:37:55,423::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1519::DEBUG::2012-12-09 12:37:55,423::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1520::DEBUG::2012-12-09 12:37:55,505::task::588::TaskManager.Task::(_updateState) Task=`c732da28-5a74-4575-9b20-9d6044075133`::moving from state init -> state preparing
>Thread-1520::INFO::2012-12-09 12:37:55,506::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1520::INFO::2012-12-09 12:37:55,506::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163748264313', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1520::DEBUG::2012-12-09 12:37:55,506::task::1172::TaskManager.Task::(prepare) Task=`c732da28-5a74-4575-9b20-9d6044075133`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163748264313', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1520::DEBUG::2012-12-09 12:37:55,506::task::588::TaskManager.Task::(_updateState) Task=`c732da28-5a74-4575-9b20-9d6044075133`::moving from state preparing -> state finished
>Thread-1520::DEBUG::2012-12-09 12:37:55,506::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1520::DEBUG::2012-12-09 12:37:55,507::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1520::DEBUG::2012-12-09 12:37:55,507::task::978::TaskManager.Task::(_decref) Task=`c732da28-5a74-4575-9b20-9d6044075133`::ref 0 aborting False
>Thread-1519::DEBUG::2012-12-09 12:37:55,525::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1519::WARNING::2012-12-09 12:37:55,526::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1519::DEBUG::2012-12-09 12:37:55,526::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1519::WARNING::2012-12-09 12:37:55,528::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1519::INFO::2012-12-09 12:37:55,529::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1519::DEBUG::2012-12-09 12:37:55,530::task::1172::TaskManager.Task::(prepare) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1519::DEBUG::2012-12-09 12:37:55,530::task::588::TaskManager.Task::(_updateState) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::moving from state preparing -> state finished
>Thread-1519::DEBUG::2012-12-09 12:37:55,530::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1519::DEBUG::2012-12-09 12:37:55,531::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1519::DEBUG::2012-12-09 12:37:55,531::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1519::DEBUG::2012-12-09 12:37:55,531::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1519::DEBUG::2012-12-09 12:37:55,531::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1519::DEBUG::2012-12-09 12:37:55,532::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1519::DEBUG::2012-12-09 12:37:55,532::task::978::TaskManager.Task::(_decref) Task=`dad339fb-6a76-47ae-8d08-98ee8113087b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:37:56,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:56,651::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016633 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:37:58,658::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:37:58,708::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0253811 s, 40.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:37:59,245::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:37:59,260::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000847794 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:00,715::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:00,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279347 s, 36.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:02,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:02,801::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:02,801::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:02,830::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0312168 s, 32.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:02,903::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:02,904::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:02,904::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:02,905::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:04,837::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:04,878::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189129 s, 54.1 MB/s\n'; <rc> = 0
>Thread-1526::DEBUG::2012-12-09 12:38:05,735::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1526::DEBUG::2012-12-09 12:38:05,736::task::588::TaskManager.Task::(_updateState) Task=`12ffedf4-82e9-464f-a723-61303c199cca`::moving from state init -> state preparing
>Thread-1526::INFO::2012-12-09 12:38:05,736::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1526::INFO::2012-12-09 12:38:05,736::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1526::DEBUG::2012-12-09 12:38:05,737::task::1172::TaskManager.Task::(prepare) Task=`12ffedf4-82e9-464f-a723-61303c199cca`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1526::DEBUG::2012-12-09 12:38:05,737::task::588::TaskManager.Task::(_updateState) Task=`12ffedf4-82e9-464f-a723-61303c199cca`::moving from state preparing -> state finished
>Thread-1526::DEBUG::2012-12-09 12:38:05,737::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1526::DEBUG::2012-12-09 12:38:05,737::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1526::DEBUG::2012-12-09 12:38:05,738::task::978::TaskManager.Task::(_decref) Task=`12ffedf4-82e9-464f-a723-61303c199cca`::ref 0 aborting False
>Thread-1527::DEBUG::2012-12-09 12:38:05,749::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1527::DEBUG::2012-12-09 12:38:05,750::task::588::TaskManager.Task::(_updateState) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::moving from state init -> state preparing
>Thread-1527::INFO::2012-12-09 12:38:05,750::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1527::DEBUG::2012-12-09 12:38:05,750::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dd029209-7804-4a0e-8237-87b2c645b371`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1527::DEBUG::2012-12-09 12:38:05,751::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1527::DEBUG::2012-12-09 12:38:05,751::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1527::DEBUG::2012-12-09 12:38:05,751::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dd029209-7804-4a0e-8237-87b2c645b371`::Granted request
>Thread-1527::DEBUG::2012-12-09 12:38:05,751::task::817::TaskManager.Task::(resourceAcquired) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1527::DEBUG::2012-12-09 12:38:05,752::task::978::TaskManager.Task::(_decref) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::ref 1 aborting False
>Thread-1527::DEBUG::2012-12-09 12:38:05,752::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1527::DEBUG::2012-12-09 12:38:05,753::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1527::DEBUG::2012-12-09 12:38:05,853::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1527::WARNING::2012-12-09 12:38:05,855::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1527::DEBUG::2012-12-09 12:38:05,855::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1527::WARNING::2012-12-09 12:38:05,856::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1527::INFO::2012-12-09 12:38:05,857::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1527::DEBUG::2012-12-09 12:38:05,857::task::1172::TaskManager.Task::(prepare) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1527::DEBUG::2012-12-09 12:38:05,858::task::588::TaskManager.Task::(_updateState) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::moving from state preparing -> state finished
>Thread-1527::DEBUG::2012-12-09 12:38:05,858::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1527::DEBUG::2012-12-09 12:38:05,859::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1527::DEBUG::2012-12-09 12:38:05,859::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1527::DEBUG::2012-12-09 12:38:05,859::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1527::DEBUG::2012-12-09 12:38:05,859::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1527::DEBUG::2012-12-09 12:38:05,860::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1527::DEBUG::2012-12-09 12:38:05,860::task::978::TaskManager.Task::(_decref) Task=`b0d0bfae-ec5f-407c-9752-64e0fe2cb191`::ref 0 aborting False
>Thread-1528::DEBUG::2012-12-09 12:38:05,928::task::588::TaskManager.Task::(_updateState) Task=`78a59ec5-c373-4d46-9edc-c5ca1379e04d`::moving from state init -> state preparing
>Thread-1528::INFO::2012-12-09 12:38:05,929::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1528::INFO::2012-12-09 12:38:05,929::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160398483276', 'lastCheck': '6.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-1528::DEBUG::2012-12-09 12:38:05,930::task::1172::TaskManager.Task::(prepare) Task=`78a59ec5-c373-4d46-9edc-c5ca1379e04d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160398483276', 'lastCheck': '6.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-1528::DEBUG::2012-12-09 12:38:05,930::task::588::TaskManager.Task::(_updateState) Task=`78a59ec5-c373-4d46-9edc-c5ca1379e04d`::moving from state preparing -> state finished
>Thread-1528::DEBUG::2012-12-09 12:38:05,930::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1528::DEBUG::2012-12-09 12:38:05,931::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1528::DEBUG::2012-12-09 12:38:05,931::task::978::TaskManager.Task::(_decref) Task=`78a59ec5-c373-4d46-9edc-c5ca1379e04d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:06,885::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:06,931::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214314 s, 47.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:08,938::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:08,976::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153892 s, 66.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:09,266::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:09,278::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000763893 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:10,983::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:11,022::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166667 s, 61.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:12,906::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:12,907::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:13,020::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:13,022::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:13,022::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:13,023::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:13,029::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:13,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250964 s, 40.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:15,085::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:15,132::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221445 s, 46.2 MB/s\n'; <rc> = 0
>Thread-1534::DEBUG::2012-12-09 12:38:16,356::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1534::DEBUG::2012-12-09 12:38:16,357::task::588::TaskManager.Task::(_updateState) Task=`d17a6e7f-1812-4151-a4bc-d438743cfb45`::moving from state init -> state preparing
>Thread-1534::INFO::2012-12-09 12:38:16,357::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1534::INFO::2012-12-09 12:38:16,358::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1534::DEBUG::2012-12-09 12:38:16,358::task::1172::TaskManager.Task::(prepare) Task=`d17a6e7f-1812-4151-a4bc-d438743cfb45`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1534::DEBUG::2012-12-09 12:38:16,359::task::588::TaskManager.Task::(_updateState) Task=`d17a6e7f-1812-4151-a4bc-d438743cfb45`::moving from state preparing -> state finished
>Thread-1534::DEBUG::2012-12-09 12:38:16,359::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1534::DEBUG::2012-12-09 12:38:16,360::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1534::DEBUG::2012-12-09 12:38:16,360::task::978::TaskManager.Task::(_decref) Task=`d17a6e7f-1812-4151-a4bc-d438743cfb45`::ref 0 aborting False
>Thread-1535::DEBUG::2012-12-09 12:38:16,385::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1535::DEBUG::2012-12-09 12:38:16,385::task::588::TaskManager.Task::(_updateState) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::moving from state init -> state preparing
>Thread-1535::INFO::2012-12-09 12:38:16,386::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1535::DEBUG::2012-12-09 12:38:16,387::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbc1d701-653f-4f35-9abc-df1704e7d32a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1535::DEBUG::2012-12-09 12:38:16,387::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1535::DEBUG::2012-12-09 12:38:16,388::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1535::DEBUG::2012-12-09 12:38:16,388::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fbc1d701-653f-4f35-9abc-df1704e7d32a`::Granted request
>Thread-1535::DEBUG::2012-12-09 12:38:16,389::task::817::TaskManager.Task::(resourceAcquired) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1535::DEBUG::2012-12-09 12:38:16,389::task::978::TaskManager.Task::(_decref) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::ref 1 aborting False
>Thread-1535::DEBUG::2012-12-09 12:38:16,391::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1535::DEBUG::2012-12-09 12:38:16,392::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1536::DEBUG::2012-12-09 12:38:16,423::task::588::TaskManager.Task::(_updateState) Task=`95180b94-30bb-44d3-a815-87d51d98d009`::moving from state init -> state preparing
>Thread-1536::INFO::2012-12-09 12:38:16,423::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1536::INFO::2012-12-09 12:38:16,423::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013769865036', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-1536::DEBUG::2012-12-09 12:38:16,424::task::1172::TaskManager.Task::(prepare) Task=`95180b94-30bb-44d3-a815-87d51d98d009`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013769865036', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-1536::DEBUG::2012-12-09 12:38:16,424::task::588::TaskManager.Task::(_updateState) Task=`95180b94-30bb-44d3-a815-87d51d98d009`::moving from state preparing -> state finished
>Thread-1536::DEBUG::2012-12-09 12:38:16,424::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1536::DEBUG::2012-12-09 12:38:16,424::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1536::DEBUG::2012-12-09 12:38:16,424::task::978::TaskManager.Task::(_decref) Task=`95180b94-30bb-44d3-a815-87d51d98d009`::ref 0 aborting False
>Thread-1535::DEBUG::2012-12-09 12:38:16,512::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1535::WARNING::2012-12-09 12:38:16,514::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1535::DEBUG::2012-12-09 12:38:16,514::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1535::WARNING::2012-12-09 12:38:16,515::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1535::INFO::2012-12-09 12:38:16,515::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1535::DEBUG::2012-12-09 12:38:16,515::task::1172::TaskManager.Task::(prepare) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1535::DEBUG::2012-12-09 12:38:16,515::task::588::TaskManager.Task::(_updateState) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::moving from state preparing -> state finished
>Thread-1535::DEBUG::2012-12-09 12:38:16,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1535::DEBUG::2012-12-09 12:38:16,516::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1535::DEBUG::2012-12-09 12:38:16,516::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1535::DEBUG::2012-12-09 12:38:16,516::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1535::DEBUG::2012-12-09 12:38:16,516::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1535::DEBUG::2012-12-09 12:38:16,517::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1535::DEBUG::2012-12-09 12:38:16,517::task::978::TaskManager.Task::(_decref) Task=`b6f56afa-3fc7-4582-9215-40293a9c3a5f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:17,138::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:17,189::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264258 s, 38.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:19,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:19,242::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216952 s, 47.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:19,284::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:19,299::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000783184 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:21,249::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:21,294::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203225 s, 50.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:23,025::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:23,026::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:23,135::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:23,136::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:23,137::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:23,137::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:23,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:23,350::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249091 s, 41.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:25,356::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:25,411::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281219 s, 36.4 MB/s\n'; <rc> = 0
>Thread-1542::DEBUG::2012-12-09 12:38:26,629::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1542::DEBUG::2012-12-09 12:38:26,630::task::588::TaskManager.Task::(_updateState) Task=`4fa3270b-09bf-4707-84e7-64feb9340c0f`::moving from state init -> state preparing
>Thread-1542::INFO::2012-12-09 12:38:26,630::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1542::INFO::2012-12-09 12:38:26,631::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1542::DEBUG::2012-12-09 12:38:26,631::task::1172::TaskManager.Task::(prepare) Task=`4fa3270b-09bf-4707-84e7-64feb9340c0f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1542::DEBUG::2012-12-09 12:38:26,631::task::588::TaskManager.Task::(_updateState) Task=`4fa3270b-09bf-4707-84e7-64feb9340c0f`::moving from state preparing -> state finished
>Thread-1542::DEBUG::2012-12-09 12:38:26,631::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1542::DEBUG::2012-12-09 12:38:26,631::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1542::DEBUG::2012-12-09 12:38:26,631::task::978::TaskManager.Task::(_decref) Task=`4fa3270b-09bf-4707-84e7-64feb9340c0f`::ref 0 aborting False
>Thread-1543::DEBUG::2012-12-09 12:38:26,655::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1543::DEBUG::2012-12-09 12:38:26,656::task::588::TaskManager.Task::(_updateState) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::moving from state init -> state preparing
>Thread-1543::INFO::2012-12-09 12:38:26,656::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1543::DEBUG::2012-12-09 12:38:26,657::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7c939244-66f0-4692-bc3d-0c7b66298080`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1543::DEBUG::2012-12-09 12:38:26,657::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1543::DEBUG::2012-12-09 12:38:26,658::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1543::DEBUG::2012-12-09 12:38:26,658::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7c939244-66f0-4692-bc3d-0c7b66298080`::Granted request
>Thread-1543::DEBUG::2012-12-09 12:38:26,659::task::817::TaskManager.Task::(resourceAcquired) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1543::DEBUG::2012-12-09 12:38:26,659::task::978::TaskManager.Task::(_decref) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::ref 1 aborting False
>Thread-1543::DEBUG::2012-12-09 12:38:26,661::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1543::DEBUG::2012-12-09 12:38:26,662::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1544::DEBUG::2012-12-09 12:38:26,734::task::588::TaskManager.Task::(_updateState) Task=`68fa2612-218c-4851-8545-1c8bd885c2d9`::moving from state init -> state preparing
>Thread-1544::INFO::2012-12-09 12:38:26,735::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1544::INFO::2012-12-09 12:38:26,735::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158669948578', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1544::DEBUG::2012-12-09 12:38:26,735::task::1172::TaskManager.Task::(prepare) Task=`68fa2612-218c-4851-8545-1c8bd885c2d9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158669948578', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1544::DEBUG::2012-12-09 12:38:26,736::task::588::TaskManager.Task::(_updateState) Task=`68fa2612-218c-4851-8545-1c8bd885c2d9`::moving from state preparing -> state finished
>Thread-1544::DEBUG::2012-12-09 12:38:26,736::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1544::DEBUG::2012-12-09 12:38:26,736::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1544::DEBUG::2012-12-09 12:38:26,737::task::978::TaskManager.Task::(_decref) Task=`68fa2612-218c-4851-8545-1c8bd885c2d9`::ref 0 aborting False
>Thread-1543::DEBUG::2012-12-09 12:38:26,769::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1543::WARNING::2012-12-09 12:38:26,771::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1543::DEBUG::2012-12-09 12:38:26,772::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1543::WARNING::2012-12-09 12:38:26,773::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1543::INFO::2012-12-09 12:38:26,774::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1543::DEBUG::2012-12-09 12:38:26,774::task::1172::TaskManager.Task::(prepare) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1543::DEBUG::2012-12-09 12:38:26,775::task::588::TaskManager.Task::(_updateState) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::moving from state preparing -> state finished
>Thread-1543::DEBUG::2012-12-09 12:38:26,775::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1543::DEBUG::2012-12-09 12:38:26,775::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1543::DEBUG::2012-12-09 12:38:26,776::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1543::DEBUG::2012-12-09 12:38:26,777::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1543::DEBUG::2012-12-09 12:38:26,777::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1543::DEBUG::2012-12-09 12:38:26,777::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1543::DEBUG::2012-12-09 12:38:26,778::task::978::TaskManager.Task::(_decref) Task=`99c6cc7d-9083-4337-b7a0-14743bf1b1d4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:27,417::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:27,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206272 s, 49.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:29,305::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:29,317::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000971789 s, 4.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:29,468::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:29,512::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204034 s, 50.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:31,519::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:31,564::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206285 s, 49.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:33,138::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:33,138::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:33,231::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:33,232::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:33,232::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:33,233::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:33,570::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:33,613::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186051 s, 55.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:35,620::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:35,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025001 s, 41.0 MB/s\n'; <rc> = 0
>Thread-1550::DEBUG::2012-12-09 12:38:36,967::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1550::DEBUG::2012-12-09 12:38:36,968::task::588::TaskManager.Task::(_updateState) Task=`03141044-45da-4a09-8829-fbd984f00248`::moving from state init -> state preparing
>Thread-1550::INFO::2012-12-09 12:38:36,969::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1550::INFO::2012-12-09 12:38:36,969::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1550::DEBUG::2012-12-09 12:38:36,969::task::1172::TaskManager.Task::(prepare) Task=`03141044-45da-4a09-8829-fbd984f00248`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1550::DEBUG::2012-12-09 12:38:36,970::task::588::TaskManager.Task::(_updateState) Task=`03141044-45da-4a09-8829-fbd984f00248`::moving from state preparing -> state finished
>Thread-1550::DEBUG::2012-12-09 12:38:36,970::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1550::DEBUG::2012-12-09 12:38:36,970::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1550::DEBUG::2012-12-09 12:38:36,970::task::978::TaskManager.Task::(_decref) Task=`03141044-45da-4a09-8829-fbd984f00248`::ref 0 aborting False
>Thread-1551::DEBUG::2012-12-09 12:38:36,995::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1551::DEBUG::2012-12-09 12:38:36,995::task::588::TaskManager.Task::(_updateState) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::moving from state init -> state preparing
>Thread-1551::INFO::2012-12-09 12:38:36,996::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1551::DEBUG::2012-12-09 12:38:36,997::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a375883b-d327-437a-b1be-0b2daab2810b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1551::DEBUG::2012-12-09 12:38:36,997::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1551::DEBUG::2012-12-09 12:38:36,998::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1551::DEBUG::2012-12-09 12:38:36,998::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a375883b-d327-437a-b1be-0b2daab2810b`::Granted request
>Thread-1551::DEBUG::2012-12-09 12:38:36,999::task::817::TaskManager.Task::(resourceAcquired) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1551::DEBUG::2012-12-09 12:38:36,999::task::978::TaskManager.Task::(_decref) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::ref 1 aborting False
>Thread-1551::DEBUG::2012-12-09 12:38:37,001::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1551::DEBUG::2012-12-09 12:38:37,002::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1552::DEBUG::2012-12-09 12:38:37,057::task::588::TaskManager.Task::(_updateState) Task=`2114bf15-010b-4c9b-b5b8-5d88fc2ce74e`::moving from state init -> state preparing
>Thread-1552::INFO::2012-12-09 12:38:37,057::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1552::INFO::2012-12-09 12:38:37,058::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134100914001', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1552::DEBUG::2012-12-09 12:38:37,058::task::1172::TaskManager.Task::(prepare) Task=`2114bf15-010b-4c9b-b5b8-5d88fc2ce74e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134100914001', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1552::DEBUG::2012-12-09 12:38:37,058::task::588::TaskManager.Task::(_updateState) Task=`2114bf15-010b-4c9b-b5b8-5d88fc2ce74e`::moving from state preparing -> state finished
>Thread-1552::DEBUG::2012-12-09 12:38:37,058::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1552::DEBUG::2012-12-09 12:38:37,058::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1552::DEBUG::2012-12-09 12:38:37,059::task::978::TaskManager.Task::(_decref) Task=`2114bf15-010b-4c9b-b5b8-5d88fc2ce74e`::ref 0 aborting False
>Thread-1551::DEBUG::2012-12-09 12:38:37,113::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1551::WARNING::2012-12-09 12:38:37,115::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1551::DEBUG::2012-12-09 12:38:37,115::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1551::WARNING::2012-12-09 12:38:37,116::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1551::INFO::2012-12-09 12:38:37,116::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1551::DEBUG::2012-12-09 12:38:37,116::task::1172::TaskManager.Task::(prepare) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1551::DEBUG::2012-12-09 12:38:37,117::task::588::TaskManager.Task::(_updateState) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::moving from state preparing -> state finished
>Thread-1551::DEBUG::2012-12-09 12:38:37,117::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1551::DEBUG::2012-12-09 12:38:37,117::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1551::DEBUG::2012-12-09 12:38:37,117::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1551::DEBUG::2012-12-09 12:38:37,118::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1551::DEBUG::2012-12-09 12:38:37,118::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1551::DEBUG::2012-12-09 12:38:37,118::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1551::DEBUG::2012-12-09 12:38:37,119::task::978::TaskManager.Task::(_decref) Task=`ee6617b9-86d8-4414-a8b3-cfab86800af1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:37,676::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:37,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219781 s, 46.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:39,323::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:39,335::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000556063 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:39,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:39,779::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248818 s, 41.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:41,786::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:41,839::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285755 s, 35.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:43,234::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:43,235::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:43,345::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:43,346::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:43,346::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:43,347::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:43,846::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:43,898::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026617 s, 38.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:45,905::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:45,952::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226031 s, 45.3 MB/s\n'; <rc> = 0
>Thread-1558::DEBUG::2012-12-09 12:38:47,337::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1558::DEBUG::2012-12-09 12:38:47,338::task::588::TaskManager.Task::(_updateState) Task=`ceaa64d1-2823-460f-88fd-46e8a7ba2d05`::moving from state init -> state preparing
>Thread-1558::INFO::2012-12-09 12:38:47,339::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1558::INFO::2012-12-09 12:38:47,339::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1558::DEBUG::2012-12-09 12:38:47,340::task::1172::TaskManager.Task::(prepare) Task=`ceaa64d1-2823-460f-88fd-46e8a7ba2d05`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1558::DEBUG::2012-12-09 12:38:47,340::task::588::TaskManager.Task::(_updateState) Task=`ceaa64d1-2823-460f-88fd-46e8a7ba2d05`::moving from state preparing -> state finished
>Thread-1558::DEBUG::2012-12-09 12:38:47,341::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1558::DEBUG::2012-12-09 12:38:47,341::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1558::DEBUG::2012-12-09 12:38:47,341::task::978::TaskManager.Task::(_decref) Task=`ceaa64d1-2823-460f-88fd-46e8a7ba2d05`::ref 0 aborting False
>Thread-1559::DEBUG::2012-12-09 12:38:47,360::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1559::DEBUG::2012-12-09 12:38:47,361::task::588::TaskManager.Task::(_updateState) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::moving from state init -> state preparing
>Thread-1559::INFO::2012-12-09 12:38:47,361::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1559::DEBUG::2012-12-09 12:38:47,362::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a7bbac2e-64ef-45d2-afbf-0f63b4023563`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1559::DEBUG::2012-12-09 12:38:47,362::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1559::DEBUG::2012-12-09 12:38:47,363::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1559::DEBUG::2012-12-09 12:38:47,363::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a7bbac2e-64ef-45d2-afbf-0f63b4023563`::Granted request
>Thread-1559::DEBUG::2012-12-09 12:38:47,364::task::817::TaskManager.Task::(resourceAcquired) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1559::DEBUG::2012-12-09 12:38:47,364::task::978::TaskManager.Task::(_decref) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::ref 1 aborting False
>Thread-1559::DEBUG::2012-12-09 12:38:47,365::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1559::DEBUG::2012-12-09 12:38:47,366::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1560::DEBUG::2012-12-09 12:38:47,453::task::588::TaskManager.Task::(_updateState) Task=`89784cf5-cb70-4746-88c5-650ac7b91c6a`::moving from state init -> state preparing
>Thread-1560::INFO::2012-12-09 12:38:47,454::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1560::INFO::2012-12-09 12:38:47,454::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120489597321', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1560::DEBUG::2012-12-09 12:38:47,454::task::1172::TaskManager.Task::(prepare) Task=`89784cf5-cb70-4746-88c5-650ac7b91c6a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120489597321', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1560::DEBUG::2012-12-09 12:38:47,454::task::588::TaskManager.Task::(_updateState) Task=`89784cf5-cb70-4746-88c5-650ac7b91c6a`::moving from state preparing -> state finished
>Thread-1560::DEBUG::2012-12-09 12:38:47,454::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1560::DEBUG::2012-12-09 12:38:47,455::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1560::DEBUG::2012-12-09 12:38:47,455::task::978::TaskManager.Task::(_decref) Task=`89784cf5-cb70-4746-88c5-650ac7b91c6a`::ref 0 aborting False
>Thread-1559::DEBUG::2012-12-09 12:38:47,473::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1559::WARNING::2012-12-09 12:38:47,474::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1559::DEBUG::2012-12-09 12:38:47,474::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1559::WARNING::2012-12-09 12:38:47,475::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1559::INFO::2012-12-09 12:38:47,475::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1559::DEBUG::2012-12-09 12:38:47,475::task::1172::TaskManager.Task::(prepare) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1559::DEBUG::2012-12-09 12:38:47,476::task::588::TaskManager.Task::(_updateState) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::moving from state preparing -> state finished
>Thread-1559::DEBUG::2012-12-09 12:38:47,476::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1559::DEBUG::2012-12-09 12:38:47,476::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1559::DEBUG::2012-12-09 12:38:47,476::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1559::DEBUG::2012-12-09 12:38:47,476::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1559::DEBUG::2012-12-09 12:38:47,477::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1559::DEBUG::2012-12-09 12:38:47,477::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1559::DEBUG::2012-12-09 12:38:47,477::task::978::TaskManager.Task::(_decref) Task=`a291465e-bc0f-4929-88e7-9c65b39e2ac0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:47,958::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:47,996::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174399 s, 58.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:49,338::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:49,350::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000865293 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:50,003::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:50,047::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210562 s, 48.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:52,054::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:52,096::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171086 s, 59.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:38:53,347::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:38:53,348::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:38:53,446::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:38:53,447::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:38:53,447::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:38:53,448::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:38:54,103::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:54,141::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164281 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:38:56,147::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:56,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277242 s, 36.9 MB/s\n'; <rc> = 0
>Thread-1566::DEBUG::2012-12-09 12:38:57,662::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1566::DEBUG::2012-12-09 12:38:57,663::task::588::TaskManager.Task::(_updateState) Task=`816f9c9b-ae2b-4b24-9b2a-88eb72e23fbf`::moving from state init -> state preparing
>Thread-1566::INFO::2012-12-09 12:38:57,663::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1566::INFO::2012-12-09 12:38:57,664::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1566::DEBUG::2012-12-09 12:38:57,664::task::1172::TaskManager.Task::(prepare) Task=`816f9c9b-ae2b-4b24-9b2a-88eb72e23fbf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1566::DEBUG::2012-12-09 12:38:57,664::task::588::TaskManager.Task::(_updateState) Task=`816f9c9b-ae2b-4b24-9b2a-88eb72e23fbf`::moving from state preparing -> state finished
>Thread-1566::DEBUG::2012-12-09 12:38:57,664::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1566::DEBUG::2012-12-09 12:38:57,665::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1566::DEBUG::2012-12-09 12:38:57,665::task::978::TaskManager.Task::(_decref) Task=`816f9c9b-ae2b-4b24-9b2a-88eb72e23fbf`::ref 0 aborting False
>Thread-1567::DEBUG::2012-12-09 12:38:57,677::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1567::DEBUG::2012-12-09 12:38:57,678::task::588::TaskManager.Task::(_updateState) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::moving from state init -> state preparing
>Thread-1567::INFO::2012-12-09 12:38:57,678::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1567::DEBUG::2012-12-09 12:38:57,679::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`252cebf0-d880-4fac-b005-e93781df81a2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1567::DEBUG::2012-12-09 12:38:57,679::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1567::DEBUG::2012-12-09 12:38:57,680::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1567::DEBUG::2012-12-09 12:38:57,680::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`252cebf0-d880-4fac-b005-e93781df81a2`::Granted request
>Thread-1567::DEBUG::2012-12-09 12:38:57,681::task::817::TaskManager.Task::(resourceAcquired) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1567::DEBUG::2012-12-09 12:38:57,682::task::978::TaskManager.Task::(_decref) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::ref 1 aborting False
>Thread-1567::DEBUG::2012-12-09 12:38:57,682::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1567::DEBUG::2012-12-09 12:38:57,683::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1568::DEBUG::2012-12-09 12:38:57,779::task::588::TaskManager.Task::(_updateState) Task=`4e19b300-7889-4d14-865f-54ffcb086967`::moving from state init -> state preparing
>Thread-1568::INFO::2012-12-09 12:38:57,780::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1568::INFO::2012-12-09 12:38:57,780::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136468410492', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-1568::DEBUG::2012-12-09 12:38:57,780::task::1172::TaskManager.Task::(prepare) Task=`4e19b300-7889-4d14-865f-54ffcb086967`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136468410492', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-1568::DEBUG::2012-12-09 12:38:57,781::task::588::TaskManager.Task::(_updateState) Task=`4e19b300-7889-4d14-865f-54ffcb086967`::moving from state preparing -> state finished
>Thread-1568::DEBUG::2012-12-09 12:38:57,781::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1568::DEBUG::2012-12-09 12:38:57,781::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1568::DEBUG::2012-12-09 12:38:57,781::task::978::TaskManager.Task::(_decref) Task=`4e19b300-7889-4d14-865f-54ffcb086967`::ref 0 aborting False
>Thread-1567::DEBUG::2012-12-09 12:38:57,797::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1567::WARNING::2012-12-09 12:38:57,798::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1567::DEBUG::2012-12-09 12:38:57,798::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1567::WARNING::2012-12-09 12:38:57,799::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1567::INFO::2012-12-09 12:38:57,799::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1567::DEBUG::2012-12-09 12:38:57,800::task::1172::TaskManager.Task::(prepare) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1567::DEBUG::2012-12-09 12:38:57,800::task::588::TaskManager.Task::(_updateState) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::moving from state preparing -> state finished
>Thread-1567::DEBUG::2012-12-09 12:38:57,800::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1567::DEBUG::2012-12-09 12:38:57,800::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1567::DEBUG::2012-12-09 12:38:57,801::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1567::DEBUG::2012-12-09 12:38:57,801::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1567::DEBUG::2012-12-09 12:38:57,801::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1567::DEBUG::2012-12-09 12:38:57,801::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1567::DEBUG::2012-12-09 12:38:57,801::task::978::TaskManager.Task::(_decref) Task=`78bd7ed4-8043-4e53-af2f-9e6b04207ff9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:38:58,207::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:38:58,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158751 s, 64.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:38:59,356::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:38:59,371::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000834 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:00,251::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:00,300::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243579 s, 42.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:02,306::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:02,357::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256514 s, 39.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:03,449::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:03,450::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:03,548::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:03,549::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:03,549::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:03,550::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:04,363::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:04,401::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164874 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:06,407::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:06,446::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166557 s, 61.5 MB/s\n'; <rc> = 0
>Thread-1574::DEBUG::2012-12-09 12:39:08,003::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1574::DEBUG::2012-12-09 12:39:08,003::task::588::TaskManager.Task::(_updateState) Task=`fb3a5045-6623-48d9-882a-749585dec523`::moving from state init -> state preparing
>Thread-1574::INFO::2012-12-09 12:39:08,004::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1574::INFO::2012-12-09 12:39:08,004::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1574::DEBUG::2012-12-09 12:39:08,004::task::1172::TaskManager.Task::(prepare) Task=`fb3a5045-6623-48d9-882a-749585dec523`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1574::DEBUG::2012-12-09 12:39:08,004::task::588::TaskManager.Task::(_updateState) Task=`fb3a5045-6623-48d9-882a-749585dec523`::moving from state preparing -> state finished
>Thread-1574::DEBUG::2012-12-09 12:39:08,004::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1574::DEBUG::2012-12-09 12:39:08,005::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1574::DEBUG::2012-12-09 12:39:08,005::task::978::TaskManager.Task::(_decref) Task=`fb3a5045-6623-48d9-882a-749585dec523`::ref 0 aborting False
>Thread-1575::DEBUG::2012-12-09 12:39:08,019::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1575::DEBUG::2012-12-09 12:39:08,019::task::588::TaskManager.Task::(_updateState) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::moving from state init -> state preparing
>Thread-1575::INFO::2012-12-09 12:39:08,019::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1575::DEBUG::2012-12-09 12:39:08,020::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a64d9dea-4df3-46b9-812d-75fd4ab0645e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1575::DEBUG::2012-12-09 12:39:08,020::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1575::DEBUG::2012-12-09 12:39:08,020::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1575::DEBUG::2012-12-09 12:39:08,020::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a64d9dea-4df3-46b9-812d-75fd4ab0645e`::Granted request
>Thread-1575::DEBUG::2012-12-09 12:39:08,021::task::817::TaskManager.Task::(resourceAcquired) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1575::DEBUG::2012-12-09 12:39:08,021::task::978::TaskManager.Task::(_decref) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::ref 1 aborting False
>Thread-1575::DEBUG::2012-12-09 12:39:08,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1575::DEBUG::2012-12-09 12:39:08,022::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1576::DEBUG::2012-12-09 12:39:08,053::task::588::TaskManager.Task::(_updateState) Task=`ed56d3c5-d8eb-40bc-8f33-7d15e102ab9a`::moving from state init -> state preparing
>Thread-1576::INFO::2012-12-09 12:39:08,054::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1576::INFO::2012-12-09 12:39:08,054::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162229537964', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-1576::DEBUG::2012-12-09 12:39:08,055::task::1172::TaskManager.Task::(prepare) Task=`ed56d3c5-d8eb-40bc-8f33-7d15e102ab9a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162229537964', 'lastCheck': '8.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-1576::DEBUG::2012-12-09 12:39:08,055::task::588::TaskManager.Task::(_updateState) Task=`ed56d3c5-d8eb-40bc-8f33-7d15e102ab9a`::moving from state preparing -> state finished
>Thread-1576::DEBUG::2012-12-09 12:39:08,055::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1576::DEBUG::2012-12-09 12:39:08,056::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1576::DEBUG::2012-12-09 12:39:08,056::task::978::TaskManager.Task::(_decref) Task=`ed56d3c5-d8eb-40bc-8f33-7d15e102ab9a`::ref 0 aborting False
>Thread-1575::DEBUG::2012-12-09 12:39:08,128::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1575::WARNING::2012-12-09 12:39:08,129::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1575::DEBUG::2012-12-09 12:39:08,130::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1575::WARNING::2012-12-09 12:39:08,130::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1575::INFO::2012-12-09 12:39:08,131::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1575::DEBUG::2012-12-09 12:39:08,131::task::1172::TaskManager.Task::(prepare) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1575::DEBUG::2012-12-09 12:39:08,131::task::588::TaskManager.Task::(_updateState) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::moving from state preparing -> state finished
>Thread-1575::DEBUG::2012-12-09 12:39:08,131::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1575::DEBUG::2012-12-09 12:39:08,132::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1575::DEBUG::2012-12-09 12:39:08,132::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1575::DEBUG::2012-12-09 12:39:08,132::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1575::DEBUG::2012-12-09 12:39:08,132::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1575::DEBUG::2012-12-09 12:39:08,133::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1575::DEBUG::2012-12-09 12:39:08,133::task::978::TaskManager.Task::(_decref) Task=`1b421bf8-73d3-4b9e-88e3-2175865f1320`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:39:08,453::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:08,494::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193112 s, 53.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:09,376::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:39:09,376::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:39:09,377::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:09,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000517882 s, 4.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:09,386::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000517882 s, 4.0 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:39:09,387::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:39:09,387::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:39:09,387::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:39:09,387::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,387::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,388::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,388::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,388::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,388::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:09,484::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:09,485::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:39:09,486::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:39:09,486::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:39:09,486::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:39:09,487::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:39:09,487::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:09,579::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:09,579::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:09,588::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000483542 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:10,501::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:10,540::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156615 s, 65.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:12,546::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:12,593::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224876 s, 45.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:13,551::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:13,552::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:13,663::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:13,664::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:13,665::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:13,665::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:14,598::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:14,640::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171074 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:16,647::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:16,700::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275811 s, 37.1 MB/s\n'; <rc> = 0
>Thread-1582::DEBUG::2012-12-09 12:39:18,264::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1582::DEBUG::2012-12-09 12:39:18,264::task::588::TaskManager.Task::(_updateState) Task=`baead7bd-81e0-41bf-92c9-a6c414484187`::moving from state init -> state preparing
>Thread-1582::INFO::2012-12-09 12:39:18,265::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1582::INFO::2012-12-09 12:39:18,265::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1582::DEBUG::2012-12-09 12:39:18,265::task::1172::TaskManager.Task::(prepare) Task=`baead7bd-81e0-41bf-92c9-a6c414484187`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1582::DEBUG::2012-12-09 12:39:18,265::task::588::TaskManager.Task::(_updateState) Task=`baead7bd-81e0-41bf-92c9-a6c414484187`::moving from state preparing -> state finished
>Thread-1582::DEBUG::2012-12-09 12:39:18,265::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1582::DEBUG::2012-12-09 12:39:18,266::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1582::DEBUG::2012-12-09 12:39:18,266::task::978::TaskManager.Task::(_decref) Task=`baead7bd-81e0-41bf-92c9-a6c414484187`::ref 0 aborting False
>Thread-1583::DEBUG::2012-12-09 12:39:18,278::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1583::DEBUG::2012-12-09 12:39:18,278::task::588::TaskManager.Task::(_updateState) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::moving from state init -> state preparing
>Thread-1583::INFO::2012-12-09 12:39:18,278::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1583::DEBUG::2012-12-09 12:39:18,279::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`85be0882-b2c7-443c-9c72-5245892544e5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1583::DEBUG::2012-12-09 12:39:18,279::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1583::DEBUG::2012-12-09 12:39:18,279::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1583::DEBUG::2012-12-09 12:39:18,279::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`85be0882-b2c7-443c-9c72-5245892544e5`::Granted request
>Thread-1583::DEBUG::2012-12-09 12:39:18,280::task::817::TaskManager.Task::(resourceAcquired) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1583::DEBUG::2012-12-09 12:39:18,280::task::978::TaskManager.Task::(_decref) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::ref 1 aborting False
>Thread-1583::DEBUG::2012-12-09 12:39:18,281::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1583::DEBUG::2012-12-09 12:39:18,281::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1584::DEBUG::2012-12-09 12:39:18,375::task::588::TaskManager.Task::(_updateState) Task=`bf6a1a35-ab13-4889-8988-6f2984bdb430`::moving from state init -> state preparing
>Thread-1584::INFO::2012-12-09 12:39:18,375::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1584::INFO::2012-12-09 12:39:18,376::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00886988639832', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1584::DEBUG::2012-12-09 12:39:18,376::task::1172::TaskManager.Task::(prepare) Task=`bf6a1a35-ab13-4889-8988-6f2984bdb430`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00886988639832', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1584::DEBUG::2012-12-09 12:39:18,376::task::588::TaskManager.Task::(_updateState) Task=`bf6a1a35-ab13-4889-8988-6f2984bdb430`::moving from state preparing -> state finished
>Thread-1584::DEBUG::2012-12-09 12:39:18,376::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1584::DEBUG::2012-12-09 12:39:18,377::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1584::DEBUG::2012-12-09 12:39:18,377::task::978::TaskManager.Task::(_decref) Task=`bf6a1a35-ab13-4889-8988-6f2984bdb430`::ref 0 aborting False
>Thread-1583::DEBUG::2012-12-09 12:39:18,383::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1583::WARNING::2012-12-09 12:39:18,385::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1583::DEBUG::2012-12-09 12:39:18,386::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1583::WARNING::2012-12-09 12:39:18,386::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1583::INFO::2012-12-09 12:39:18,387::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1583::DEBUG::2012-12-09 12:39:18,387::task::1172::TaskManager.Task::(prepare) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1583::DEBUG::2012-12-09 12:39:18,387::task::588::TaskManager.Task::(_updateState) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::moving from state preparing -> state finished
>Thread-1583::DEBUG::2012-12-09 12:39:18,387::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1583::DEBUG::2012-12-09 12:39:18,388::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1583::DEBUG::2012-12-09 12:39:18,388::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1583::DEBUG::2012-12-09 12:39:18,388::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1583::DEBUG::2012-12-09 12:39:18,388::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1583::DEBUG::2012-12-09 12:39:18,389::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1583::DEBUG::2012-12-09 12:39:18,389::task::978::TaskManager.Task::(_decref) Task=`8cdd96b4-5bc7-43cb-a2a3-d8c6a46b5f69`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:39:18,707::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:18,752::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209982 s, 48.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:19,591::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:19,605::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000956026 s, 4.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:20,759::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:20,810::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0252821 s, 40.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:22,817::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:22,870::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269678 s, 38.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:23,666::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:39:23,667::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:23,668::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:23,794::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:23,795::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:23,795::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:23,795::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:24,877::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:24,932::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0289372 s, 35.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:26,938::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:26,991::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262968 s, 38.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:28,997::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:29,039::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184192 s, 55.6 MB/s\n'; <rc> = 0
>Thread-1590::DEBUG::2012-12-09 12:39:29,136::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1590::DEBUG::2012-12-09 12:39:29,137::task::588::TaskManager.Task::(_updateState) Task=`50ef18bd-bfe3-4483-8ced-37b396190947`::moving from state init -> state preparing
>Thread-1590::INFO::2012-12-09 12:39:29,138::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1590::INFO::2012-12-09 12:39:29,139::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1590::DEBUG::2012-12-09 12:39:29,139::task::1172::TaskManager.Task::(prepare) Task=`50ef18bd-bfe3-4483-8ced-37b396190947`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1590::DEBUG::2012-12-09 12:39:29,139::task::588::TaskManager.Task::(_updateState) Task=`50ef18bd-bfe3-4483-8ced-37b396190947`::moving from state preparing -> state finished
>Thread-1590::DEBUG::2012-12-09 12:39:29,140::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1590::DEBUG::2012-12-09 12:39:29,140::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1590::DEBUG::2012-12-09 12:39:29,141::task::978::TaskManager.Task::(_decref) Task=`50ef18bd-bfe3-4483-8ced-37b396190947`::ref 0 aborting False
>Thread-1591::DEBUG::2012-12-09 12:39:29,165::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1591::DEBUG::2012-12-09 12:39:29,166::task::588::TaskManager.Task::(_updateState) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::moving from state init -> state preparing
>Thread-1591::INFO::2012-12-09 12:39:29,166::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1591::DEBUG::2012-12-09 12:39:29,167::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5aecc16a-9b84-4688-bbb1-a233ea9116b0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1591::DEBUG::2012-12-09 12:39:29,167::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1591::DEBUG::2012-12-09 12:39:29,168::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1591::DEBUG::2012-12-09 12:39:29,168::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5aecc16a-9b84-4688-bbb1-a233ea9116b0`::Granted request
>Thread-1591::DEBUG::2012-12-09 12:39:29,169::task::817::TaskManager.Task::(resourceAcquired) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1591::DEBUG::2012-12-09 12:39:29,170::task::978::TaskManager.Task::(_decref) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::ref 1 aborting False
>Thread-1591::DEBUG::2012-12-09 12:39:29,171::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1591::DEBUG::2012-12-09 12:39:29,172::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1592::DEBUG::2012-12-09 12:39:29,193::task::588::TaskManager.Task::(_updateState) Task=`9fd6c2aa-5fe2-4f7e-af60-991311117126`::moving from state init -> state preparing
>Thread-1592::INFO::2012-12-09 12:39:29,194::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1592::INFO::2012-12-09 12:39:29,194::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0150458812714', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1592::DEBUG::2012-12-09 12:39:29,194::task::1172::TaskManager.Task::(prepare) Task=`9fd6c2aa-5fe2-4f7e-af60-991311117126`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0150458812714', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1592::DEBUG::2012-12-09 12:39:29,195::task::588::TaskManager.Task::(_updateState) Task=`9fd6c2aa-5fe2-4f7e-af60-991311117126`::moving from state preparing -> state finished
>Thread-1592::DEBUG::2012-12-09 12:39:29,195::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1592::DEBUG::2012-12-09 12:39:29,195::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1592::DEBUG::2012-12-09 12:39:29,195::task::978::TaskManager.Task::(_decref) Task=`9fd6c2aa-5fe2-4f7e-af60-991311117126`::ref 0 aborting False
>Thread-1591::DEBUG::2012-12-09 12:39:29,285::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1591::WARNING::2012-12-09 12:39:29,287::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1591::DEBUG::2012-12-09 12:39:29,288::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1591::WARNING::2012-12-09 12:39:29,289::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1591::INFO::2012-12-09 12:39:29,289::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1591::DEBUG::2012-12-09 12:39:29,290::task::1172::TaskManager.Task::(prepare) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1591::DEBUG::2012-12-09 12:39:29,290::task::588::TaskManager.Task::(_updateState) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::moving from state preparing -> state finished
>Thread-1591::DEBUG::2012-12-09 12:39:29,291::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1591::DEBUG::2012-12-09 12:39:29,291::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1591::DEBUG::2012-12-09 12:39:29,292::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1591::DEBUG::2012-12-09 12:39:29,292::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1591::DEBUG::2012-12-09 12:39:29,293::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1591::DEBUG::2012-12-09 12:39:29,293::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1591::DEBUG::2012-12-09 12:39:29,294::task::978::TaskManager.Task::(_decref) Task=`ebeb2165-613f-4d6f-a962-cf4e87ced0e4`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:39:29,608::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:29,621::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000861794 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:31,046::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:31,098::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272099 s, 37.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:33,104::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:33,153::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023108 s, 44.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:33,796::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:33,797::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:33,895::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:33,896::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:33,897::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:33,897::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:35,160::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:35,207::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204274 s, 50.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:37,213::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:37,261::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245653 s, 41.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:39,267::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:39,313::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221257 s, 46.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:39,626::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:39,641::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000793368 s, 5.2 MB/s\n'; <rc> = 0
>Thread-1598::DEBUG::2012-12-09 12:39:39,734::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1598::DEBUG::2012-12-09 12:39:39,735::task::588::TaskManager.Task::(_updateState) Task=`ba24a450-1d77-4a94-b45b-590c8e880f69`::moving from state init -> state preparing
>Thread-1598::INFO::2012-12-09 12:39:39,735::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1598::INFO::2012-12-09 12:39:39,736::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1598::DEBUG::2012-12-09 12:39:39,736::task::1172::TaskManager.Task::(prepare) Task=`ba24a450-1d77-4a94-b45b-590c8e880f69`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1598::DEBUG::2012-12-09 12:39:39,736::task::588::TaskManager.Task::(_updateState) Task=`ba24a450-1d77-4a94-b45b-590c8e880f69`::moving from state preparing -> state finished
>Thread-1598::DEBUG::2012-12-09 12:39:39,736::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1598::DEBUG::2012-12-09 12:39:39,736::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1598::DEBUG::2012-12-09 12:39:39,737::task::978::TaskManager.Task::(_decref) Task=`ba24a450-1d77-4a94-b45b-590c8e880f69`::ref 0 aborting False
>Thread-1599::DEBUG::2012-12-09 12:39:39,755::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1599::DEBUG::2012-12-09 12:39:39,756::task::588::TaskManager.Task::(_updateState) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::moving from state init -> state preparing
>Thread-1599::INFO::2012-12-09 12:39:39,756::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1599::DEBUG::2012-12-09 12:39:39,756::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`796d5b01-86b2-4670-80a8-7f121873d1f5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1599::DEBUG::2012-12-09 12:39:39,757::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1599::DEBUG::2012-12-09 12:39:39,757::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1599::DEBUG::2012-12-09 12:39:39,757::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`796d5b01-86b2-4670-80a8-7f121873d1f5`::Granted request
>Thread-1599::DEBUG::2012-12-09 12:39:39,757::task::817::TaskManager.Task::(resourceAcquired) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1599::DEBUG::2012-12-09 12:39:39,757::task::978::TaskManager.Task::(_decref) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::ref 1 aborting False
>Thread-1599::DEBUG::2012-12-09 12:39:39,758::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1599::DEBUG::2012-12-09 12:39:39,759::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1600::DEBUG::2012-12-09 12:39:39,811::task::588::TaskManager.Task::(_updateState) Task=`c7f7489e-7600-4b29-8345-cf0ffb6d7bb0`::moving from state init -> state preparing
>Thread-1600::INFO::2012-12-09 12:39:39,811::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1600::INFO::2012-12-09 12:39:39,812::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158820152283', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-1600::DEBUG::2012-12-09 12:39:39,812::task::1172::TaskManager.Task::(prepare) Task=`c7f7489e-7600-4b29-8345-cf0ffb6d7bb0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0158820152283', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-1600::DEBUG::2012-12-09 12:39:39,812::task::588::TaskManager.Task::(_updateState) Task=`c7f7489e-7600-4b29-8345-cf0ffb6d7bb0`::moving from state preparing -> state finished
>Thread-1600::DEBUG::2012-12-09 12:39:39,813::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1600::DEBUG::2012-12-09 12:39:39,813::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1600::DEBUG::2012-12-09 12:39:39,813::task::978::TaskManager.Task::(_decref) Task=`c7f7489e-7600-4b29-8345-cf0ffb6d7bb0`::ref 0 aborting False
>Thread-1599::DEBUG::2012-12-09 12:39:39,869::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1599::WARNING::2012-12-09 12:39:39,870::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1599::DEBUG::2012-12-09 12:39:39,870::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1599::WARNING::2012-12-09 12:39:39,870::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1599::INFO::2012-12-09 12:39:39,871::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1599::DEBUG::2012-12-09 12:39:39,871::task::1172::TaskManager.Task::(prepare) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1599::DEBUG::2012-12-09 12:39:39,871::task::588::TaskManager.Task::(_updateState) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::moving from state preparing -> state finished
>Thread-1599::DEBUG::2012-12-09 12:39:39,871::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1599::DEBUG::2012-12-09 12:39:39,872::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1599::DEBUG::2012-12-09 12:39:39,872::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1599::DEBUG::2012-12-09 12:39:39,872::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1599::DEBUG::2012-12-09 12:39:39,872::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1599::DEBUG::2012-12-09 12:39:39,872::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1599::DEBUG::2012-12-09 12:39:39,873::task::978::TaskManager.Task::(_decref) Task=`20e1a724-8c61-48a5-8ebe-ca62b52a48ae`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:39:41,320::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:41,363::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189373 s, 54.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:43,370::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:43,420::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251348 s, 40.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:43,898::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:43,899::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:44,017::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:44,018::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:44,019::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:44,019::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:45,427::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:45,478::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263219 s, 38.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:47,484::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:47,544::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0341669 s, 30.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:49,551::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:49,599::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0252959 s, 40.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:49,646::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:49,660::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000812027 s, 5.0 MB/s\n'; <rc> = 0
>Thread-1606::DEBUG::2012-12-09 12:39:49,961::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1606::DEBUG::2012-12-09 12:39:49,962::task::588::TaskManager.Task::(_updateState) Task=`c9b84a3a-d031-43b5-b64d-c559ba73c93c`::moving from state init -> state preparing
>Thread-1606::INFO::2012-12-09 12:39:49,962::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1606::INFO::2012-12-09 12:39:49,963::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1606::DEBUG::2012-12-09 12:39:49,963::task::1172::TaskManager.Task::(prepare) Task=`c9b84a3a-d031-43b5-b64d-c559ba73c93c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1606::DEBUG::2012-12-09 12:39:49,964::task::588::TaskManager.Task::(_updateState) Task=`c9b84a3a-d031-43b5-b64d-c559ba73c93c`::moving from state preparing -> state finished
>Thread-1606::DEBUG::2012-12-09 12:39:49,964::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1606::DEBUG::2012-12-09 12:39:49,964::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1606::DEBUG::2012-12-09 12:39:49,965::task::978::TaskManager.Task::(_decref) Task=`c9b84a3a-d031-43b5-b64d-c559ba73c93c`::ref 0 aborting False
>Thread-1607::DEBUG::2012-12-09 12:39:49,981::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1607::DEBUG::2012-12-09 12:39:49,982::task::588::TaskManager.Task::(_updateState) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::moving from state init -> state preparing
>Thread-1607::INFO::2012-12-09 12:39:49,983::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1607::DEBUG::2012-12-09 12:39:49,983::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`11352159-cd63-4782-bae5-99e023a0a802`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1607::DEBUG::2012-12-09 12:39:49,984::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1607::DEBUG::2012-12-09 12:39:49,984::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1607::DEBUG::2012-12-09 12:39:49,985::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`11352159-cd63-4782-bae5-99e023a0a802`::Granted request
>Thread-1607::DEBUG::2012-12-09 12:39:49,985::task::817::TaskManager.Task::(resourceAcquired) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1607::DEBUG::2012-12-09 12:39:49,985::task::978::TaskManager.Task::(_decref) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::ref 1 aborting False
>Thread-1607::DEBUG::2012-12-09 12:39:49,986::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1607::DEBUG::2012-12-09 12:39:49,986::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1608::DEBUG::2012-12-09 12:39:50,060::task::588::TaskManager.Task::(_updateState) Task=`6e796a1e-6c86-4c13-9605-9f5dda829874`::moving from state init -> state preparing
>Thread-1608::INFO::2012-12-09 12:39:50,061::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1608::INFO::2012-12-09 12:39:50,061::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149960517883', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-1608::DEBUG::2012-12-09 12:39:50,061::task::1172::TaskManager.Task::(prepare) Task=`6e796a1e-6c86-4c13-9605-9f5dda829874`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149960517883', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-1608::DEBUG::2012-12-09 12:39:50,061::task::588::TaskManager.Task::(_updateState) Task=`6e796a1e-6c86-4c13-9605-9f5dda829874`::moving from state preparing -> state finished
>Thread-1608::DEBUG::2012-12-09 12:39:50,062::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1608::DEBUG::2012-12-09 12:39:50,062::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1608::DEBUG::2012-12-09 12:39:50,062::task::978::TaskManager.Task::(_decref) Task=`6e796a1e-6c86-4c13-9605-9f5dda829874`::ref 0 aborting False
>Thread-1607::DEBUG::2012-12-09 12:39:50,089::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1607::WARNING::2012-12-09 12:39:50,090::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1607::DEBUG::2012-12-09 12:39:50,090::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1607::WARNING::2012-12-09 12:39:50,091::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1607::INFO::2012-12-09 12:39:50,091::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1607::DEBUG::2012-12-09 12:39:50,091::task::1172::TaskManager.Task::(prepare) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1607::DEBUG::2012-12-09 12:39:50,091::task::588::TaskManager.Task::(_updateState) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::moving from state preparing -> state finished
>Thread-1607::DEBUG::2012-12-09 12:39:50,092::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1607::DEBUG::2012-12-09 12:39:50,092::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1607::DEBUG::2012-12-09 12:39:50,092::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1607::DEBUG::2012-12-09 12:39:50,092::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1607::DEBUG::2012-12-09 12:39:50,092::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1607::DEBUG::2012-12-09 12:39:50,093::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1607::DEBUG::2012-12-09 12:39:50,093::task::978::TaskManager.Task::(_decref) Task=`e7866730-bf08-451c-ba08-3f53531b3ddf`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:39:51,606::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:51,659::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275071 s, 37.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:53,666::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:53,712::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021704 s, 47.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:39:54,020::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:39:54,021::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:39:54,132::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:39:54,134::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:39:54,134::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:39:54,135::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:39:55,719::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:55,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193121 s, 53.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:57,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:57,823::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229756 s, 44.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:39:59,666::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:39:59,682::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000856534 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:39:59,829::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:39:59,880::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255506 s, 40.1 MB/s\n'; <rc> = 0
>Thread-1614::DEBUG::2012-12-09 12:40:00,240::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1614::DEBUG::2012-12-09 12:40:00,241::task::588::TaskManager.Task::(_updateState) Task=`3caf17f3-5dc6-4c48-ae7b-9401c789f6d1`::moving from state init -> state preparing
>Thread-1614::INFO::2012-12-09 12:40:00,241::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1614::INFO::2012-12-09 12:40:00,242::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1614::DEBUG::2012-12-09 12:40:00,242::task::1172::TaskManager.Task::(prepare) Task=`3caf17f3-5dc6-4c48-ae7b-9401c789f6d1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1614::DEBUG::2012-12-09 12:40:00,243::task::588::TaskManager.Task::(_updateState) Task=`3caf17f3-5dc6-4c48-ae7b-9401c789f6d1`::moving from state preparing -> state finished
>Thread-1614::DEBUG::2012-12-09 12:40:00,243::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1614::DEBUG::2012-12-09 12:40:00,244::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1614::DEBUG::2012-12-09 12:40:00,244::task::978::TaskManager.Task::(_decref) Task=`3caf17f3-5dc6-4c48-ae7b-9401c789f6d1`::ref 0 aborting False
>Thread-1615::DEBUG::2012-12-09 12:40:00,267::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1615::DEBUG::2012-12-09 12:40:00,267::task::588::TaskManager.Task::(_updateState) Task=`0aec344d-e335-474e-8faf-6d931358a938`::moving from state init -> state preparing
>Thread-1615::INFO::2012-12-09 12:40:00,268::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1615::DEBUG::2012-12-09 12:40:00,269::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`64770e04-efc8-47ea-9ef1-888dcc8cb7bd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1615::DEBUG::2012-12-09 12:40:00,269::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1615::DEBUG::2012-12-09 12:40:00,269::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1615::DEBUG::2012-12-09 12:40:00,270::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`64770e04-efc8-47ea-9ef1-888dcc8cb7bd`::Granted request
>Thread-1615::DEBUG::2012-12-09 12:40:00,270::task::817::TaskManager.Task::(resourceAcquired) Task=`0aec344d-e335-474e-8faf-6d931358a938`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1615::DEBUG::2012-12-09 12:40:00,270::task::978::TaskManager.Task::(_decref) Task=`0aec344d-e335-474e-8faf-6d931358a938`::ref 1 aborting False
>Thread-1615::DEBUG::2012-12-09 12:40:00,272::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1615::DEBUG::2012-12-09 12:40:00,273::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1616::DEBUG::2012-12-09 12:40:00,342::task::588::TaskManager.Task::(_updateState) Task=`b9f2ce9e-f562-406a-9138-5001865fb93a`::moving from state init -> state preparing
>Thread-1616::INFO::2012-12-09 12:40:00,343::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1616::INFO::2012-12-09 12:40:00,343::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169999599457', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-1616::DEBUG::2012-12-09 12:40:00,343::task::1172::TaskManager.Task::(prepare) Task=`b9f2ce9e-f562-406a-9138-5001865fb93a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0169999599457', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-1616::DEBUG::2012-12-09 12:40:00,344::task::588::TaskManager.Task::(_updateState) Task=`b9f2ce9e-f562-406a-9138-5001865fb93a`::moving from state preparing -> state finished
>Thread-1616::DEBUG::2012-12-09 12:40:00,344::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1616::DEBUG::2012-12-09 12:40:00,344::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1616::DEBUG::2012-12-09 12:40:00,344::task::978::TaskManager.Task::(_decref) Task=`b9f2ce9e-f562-406a-9138-5001865fb93a`::ref 0 aborting False
>Thread-1615::DEBUG::2012-12-09 12:40:00,388::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1615::WARNING::2012-12-09 12:40:00,390::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1615::DEBUG::2012-12-09 12:40:00,390::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1615::WARNING::2012-12-09 12:40:00,391::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1615::INFO::2012-12-09 12:40:00,392::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1615::DEBUG::2012-12-09 12:40:00,392::task::1172::TaskManager.Task::(prepare) Task=`0aec344d-e335-474e-8faf-6d931358a938`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1615::DEBUG::2012-12-09 12:40:00,392::task::588::TaskManager.Task::(_updateState) Task=`0aec344d-e335-474e-8faf-6d931358a938`::moving from state preparing -> state finished
>Thread-1615::DEBUG::2012-12-09 12:40:00,393::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1615::DEBUG::2012-12-09 12:40:00,393::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1615::DEBUG::2012-12-09 12:40:00,393::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1615::DEBUG::2012-12-09 12:40:00,394::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1615::DEBUG::2012-12-09 12:40:00,394::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1615::DEBUG::2012-12-09 12:40:00,394::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1615::DEBUG::2012-12-09 12:40:00,395::task::978::TaskManager.Task::(_decref) Task=`0aec344d-e335-474e-8faf-6d931358a938`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:01,886::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:01,939::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274451 s, 37.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:03,946::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:03,990::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192673 s, 53.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:04,136::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:04,137::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:04,248::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:04,250::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:04,250::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:04,251::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:05,997::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:06,048::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266784 s, 38.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:08,053::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:08,095::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184381 s, 55.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:09,687::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:09,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00105661 s, 3.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:10,101::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:10,147::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209574 s, 48.9 MB/s\n'; <rc> = 0
>Thread-1622::DEBUG::2012-12-09 12:40:10,801::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1622::DEBUG::2012-12-09 12:40:10,802::task::588::TaskManager.Task::(_updateState) Task=`84882e85-c30e-427e-9bfe-5e1323360bc1`::moving from state init -> state preparing
>Thread-1622::INFO::2012-12-09 12:40:10,803::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1622::INFO::2012-12-09 12:40:10,803::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1622::DEBUG::2012-12-09 12:40:10,804::task::1172::TaskManager.Task::(prepare) Task=`84882e85-c30e-427e-9bfe-5e1323360bc1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1622::DEBUG::2012-12-09 12:40:10,804::task::588::TaskManager.Task::(_updateState) Task=`84882e85-c30e-427e-9bfe-5e1323360bc1`::moving from state preparing -> state finished
>Thread-1622::DEBUG::2012-12-09 12:40:10,805::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1622::DEBUG::2012-12-09 12:40:10,805::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1622::DEBUG::2012-12-09 12:40:10,805::task::978::TaskManager.Task::(_decref) Task=`84882e85-c30e-427e-9bfe-5e1323360bc1`::ref 0 aborting False
>Thread-1623::DEBUG::2012-12-09 12:40:10,835::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1623::DEBUG::2012-12-09 12:40:10,836::task::588::TaskManager.Task::(_updateState) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::moving from state init -> state preparing
>Thread-1623::INFO::2012-12-09 12:40:10,836::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1623::DEBUG::2012-12-09 12:40:10,837::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bd33eb02-9145-4a68-a169-cb8039d2a52d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1623::DEBUG::2012-12-09 12:40:10,838::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1623::DEBUG::2012-12-09 12:40:10,838::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1623::DEBUG::2012-12-09 12:40:10,839::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bd33eb02-9145-4a68-a169-cb8039d2a52d`::Granted request
>Thread-1623::DEBUG::2012-12-09 12:40:10,839::task::817::TaskManager.Task::(resourceAcquired) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1623::DEBUG::2012-12-09 12:40:10,840::task::978::TaskManager.Task::(_decref) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::ref 1 aborting False
>Thread-1623::DEBUG::2012-12-09 12:40:10,841::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1623::DEBUG::2012-12-09 12:40:10,842::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1624::DEBUG::2012-12-09 12:40:10,942::task::588::TaskManager.Task::(_updateState) Task=`63e4be3a-a66a-4732-a0ce-759e76a5e1fd`::moving from state init -> state preparing
>Thread-1624::INFO::2012-12-09 12:40:10,943::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1624::INFO::2012-12-09 12:40:10,943::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136439800262', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-1624::DEBUG::2012-12-09 12:40:10,943::task::1172::TaskManager.Task::(prepare) Task=`63e4be3a-a66a-4732-a0ce-759e76a5e1fd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136439800262', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-1624::DEBUG::2012-12-09 12:40:10,943::task::588::TaskManager.Task::(_updateState) Task=`63e4be3a-a66a-4732-a0ce-759e76a5e1fd`::moving from state preparing -> state finished
>Thread-1624::DEBUG::2012-12-09 12:40:10,944::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1623::DEBUG::2012-12-09 12:40:10,944::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1623::WARNING::2012-12-09 12:40:10,945::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1623::DEBUG::2012-12-09 12:40:10,945::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1623::WARNING::2012-12-09 12:40:10,946::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1623::INFO::2012-12-09 12:40:10,946::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1623::DEBUG::2012-12-09 12:40:10,946::task::1172::TaskManager.Task::(prepare) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1623::DEBUG::2012-12-09 12:40:10,947::task::588::TaskManager.Task::(_updateState) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::moving from state preparing -> state finished
>Thread-1623::DEBUG::2012-12-09 12:40:10,947::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1623::DEBUG::2012-12-09 12:40:10,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1623::DEBUG::2012-12-09 12:40:10,947::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1623::DEBUG::2012-12-09 12:40:10,947::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1623::DEBUG::2012-12-09 12:40:10,948::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1623::DEBUG::2012-12-09 12:40:10,948::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1623::DEBUG::2012-12-09 12:40:10,948::task::978::TaskManager.Task::(_decref) Task=`79bb068f-5cda-488d-9de4-b99f72a09caa`::ref 0 aborting False
>Thread-1624::DEBUG::2012-12-09 12:40:10,944::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1624::DEBUG::2012-12-09 12:40:10,949::task::978::TaskManager.Task::(_decref) Task=`63e4be3a-a66a-4732-a0ce-759e76a5e1fd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:12,154::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:12,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227935 s, 44.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:14,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:14,252::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:14,253::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:14,260::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211906 s, 48.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:14,351::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:14,352::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:14,353::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:14,353::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:16,270::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:16,320::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247089 s, 41.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:18,326::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:18,379::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280894 s, 36.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:19,705::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:19,721::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000796177 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:20,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:20,438::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275151 s, 37.2 MB/s\n'; <rc> = 0
>Thread-1630::DEBUG::2012-12-09 12:40:21,079::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1630::DEBUG::2012-12-09 12:40:21,080::task::588::TaskManager.Task::(_updateState) Task=`d8567d96-a7d0-41a5-98a2-c8dbefd345b5`::moving from state init -> state preparing
>Thread-1630::INFO::2012-12-09 12:40:21,081::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1630::INFO::2012-12-09 12:40:21,081::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1630::DEBUG::2012-12-09 12:40:21,082::task::1172::TaskManager.Task::(prepare) Task=`d8567d96-a7d0-41a5-98a2-c8dbefd345b5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1630::DEBUG::2012-12-09 12:40:21,082::task::588::TaskManager.Task::(_updateState) Task=`d8567d96-a7d0-41a5-98a2-c8dbefd345b5`::moving from state preparing -> state finished
>Thread-1630::DEBUG::2012-12-09 12:40:21,083::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1630::DEBUG::2012-12-09 12:40:21,083::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1630::DEBUG::2012-12-09 12:40:21,083::task::978::TaskManager.Task::(_decref) Task=`d8567d96-a7d0-41a5-98a2-c8dbefd345b5`::ref 0 aborting False
>Thread-1631::DEBUG::2012-12-09 12:40:21,107::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1631::DEBUG::2012-12-09 12:40:21,108::task::588::TaskManager.Task::(_updateState) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::moving from state init -> state preparing
>Thread-1631::INFO::2012-12-09 12:40:21,108::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1631::DEBUG::2012-12-09 12:40:21,109::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d50747bf-d7e9-40a1-bcd1-b20c0b2adf7d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1631::DEBUG::2012-12-09 12:40:21,109::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1631::DEBUG::2012-12-09 12:40:21,110::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1631::DEBUG::2012-12-09 12:40:21,110::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d50747bf-d7e9-40a1-bcd1-b20c0b2adf7d`::Granted request
>Thread-1631::DEBUG::2012-12-09 12:40:21,111::task::817::TaskManager.Task::(resourceAcquired) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1631::DEBUG::2012-12-09 12:40:21,111::task::978::TaskManager.Task::(_decref) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::ref 1 aborting False
>Thread-1631::DEBUG::2012-12-09 12:40:21,113::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1631::DEBUG::2012-12-09 12:40:21,114::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1632::DEBUG::2012-12-09 12:40:21,190::task::588::TaskManager.Task::(_updateState) Task=`2ea6c218-7068-41eb-8a1d-f79398a17576`::moving from state init -> state preparing
>Thread-1632::INFO::2012-12-09 12:40:21,190::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1632::INFO::2012-12-09 12:40:21,191::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167288780212', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.8', 'code': 358, 'valid': False}}
>Thread-1632::DEBUG::2012-12-09 12:40:21,191::task::1172::TaskManager.Task::(prepare) Task=`2ea6c218-7068-41eb-8a1d-f79398a17576`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167288780212', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.8', 'code': 358, 'valid': False}}
>Thread-1632::DEBUG::2012-12-09 12:40:21,191::task::588::TaskManager.Task::(_updateState) Task=`2ea6c218-7068-41eb-8a1d-f79398a17576`::moving from state preparing -> state finished
>Thread-1632::DEBUG::2012-12-09 12:40:21,191::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1632::DEBUG::2012-12-09 12:40:21,191::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1632::DEBUG::2012-12-09 12:40:21,192::task::978::TaskManager.Task::(_decref) Task=`2ea6c218-7068-41eb-8a1d-f79398a17576`::ref 0 aborting False
>Thread-1631::DEBUG::2012-12-09 12:40:21,215::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1631::WARNING::2012-12-09 12:40:21,216::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1631::DEBUG::2012-12-09 12:40:21,217::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1631::WARNING::2012-12-09 12:40:21,218::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1631::INFO::2012-12-09 12:40:21,218::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1631::DEBUG::2012-12-09 12:40:21,219::task::1172::TaskManager.Task::(prepare) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1631::DEBUG::2012-12-09 12:40:21,219::task::588::TaskManager.Task::(_updateState) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::moving from state preparing -> state finished
>Thread-1631::DEBUG::2012-12-09 12:40:21,220::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1631::DEBUG::2012-12-09 12:40:21,220::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1631::DEBUG::2012-12-09 12:40:21,221::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1631::DEBUG::2012-12-09 12:40:21,221::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1631::DEBUG::2012-12-09 12:40:21,222::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1631::DEBUG::2012-12-09 12:40:21,222::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1631::DEBUG::2012-12-09 12:40:21,223::task::978::TaskManager.Task::(_decref) Task=`02bdca35-47a0-409c-ada5-07cd6a7cf6e1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:22,444::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:22,491::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236692 s, 43.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:24,354::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:24,355::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:24,463::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:24,465::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:24,465::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:24,466::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:24,498::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:24,548::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261422 s, 39.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:26,555::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:26,602::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224509 s, 45.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:28,609::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:28,651::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178432 s, 57.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:29,726::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:29,742::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000925375 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:30,657::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:30,709::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268227 s, 38.2 MB/s\n'; <rc> = 0
>Thread-1638::DEBUG::2012-12-09 12:40:31,316::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1638::DEBUG::2012-12-09 12:40:31,318::task::588::TaskManager.Task::(_updateState) Task=`691c1729-5938-4bc1-9e95-5e3f06eae1e4`::moving from state init -> state preparing
>Thread-1638::INFO::2012-12-09 12:40:31,318::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1638::INFO::2012-12-09 12:40:31,319::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1638::DEBUG::2012-12-09 12:40:31,319::task::1172::TaskManager.Task::(prepare) Task=`691c1729-5938-4bc1-9e95-5e3f06eae1e4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1638::DEBUG::2012-12-09 12:40:31,320::task::588::TaskManager.Task::(_updateState) Task=`691c1729-5938-4bc1-9e95-5e3f06eae1e4`::moving from state preparing -> state finished
>Thread-1638::DEBUG::2012-12-09 12:40:31,320::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1638::DEBUG::2012-12-09 12:40:31,320::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1638::DEBUG::2012-12-09 12:40:31,321::task::978::TaskManager.Task::(_decref) Task=`691c1729-5938-4bc1-9e95-5e3f06eae1e4`::ref 0 aborting False
>Thread-1639::DEBUG::2012-12-09 12:40:31,340::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1639::DEBUG::2012-12-09 12:40:31,341::task::588::TaskManager.Task::(_updateState) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::moving from state init -> state preparing
>Thread-1639::INFO::2012-12-09 12:40:31,341::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1639::DEBUG::2012-12-09 12:40:31,342::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1ab086d-0d46-49fe-aaf0-9a5ea9f63307`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1639::DEBUG::2012-12-09 12:40:31,342::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1639::DEBUG::2012-12-09 12:40:31,343::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1639::DEBUG::2012-12-09 12:40:31,343::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1ab086d-0d46-49fe-aaf0-9a5ea9f63307`::Granted request
>Thread-1639::DEBUG::2012-12-09 12:40:31,344::task::817::TaskManager.Task::(resourceAcquired) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1639::DEBUG::2012-12-09 12:40:31,345::task::978::TaskManager.Task::(_decref) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::ref 1 aborting False
>Thread-1639::DEBUG::2012-12-09 12:40:31,346::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1639::DEBUG::2012-12-09 12:40:31,346::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1640::DEBUG::2012-12-09 12:40:31,401::task::588::TaskManager.Task::(_updateState) Task=`7c465e54-58d2-4a1c-b3e4-00be912e4745`::moving from state init -> state preparing
>Thread-1640::INFO::2012-12-09 12:40:31,402::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1640::INFO::2012-12-09 12:40:31,402::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016970872879', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1640::DEBUG::2012-12-09 12:40:31,402::task::1172::TaskManager.Task::(prepare) Task=`7c465e54-58d2-4a1c-b3e4-00be912e4745`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016970872879', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1640::DEBUG::2012-12-09 12:40:31,403::task::588::TaskManager.Task::(_updateState) Task=`7c465e54-58d2-4a1c-b3e4-00be912e4745`::moving from state preparing -> state finished
>Thread-1640::DEBUG::2012-12-09 12:40:31,403::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1640::DEBUG::2012-12-09 12:40:31,403::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1640::DEBUG::2012-12-09 12:40:31,403::task::978::TaskManager.Task::(_decref) Task=`7c465e54-58d2-4a1c-b3e4-00be912e4745`::ref 0 aborting False
>Thread-1639::DEBUG::2012-12-09 12:40:31,457::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1639::WARNING::2012-12-09 12:40:31,459::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1639::DEBUG::2012-12-09 12:40:31,460::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1639::WARNING::2012-12-09 12:40:31,461::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1639::INFO::2012-12-09 12:40:31,462::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1639::DEBUG::2012-12-09 12:40:31,462::task::1172::TaskManager.Task::(prepare) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1639::DEBUG::2012-12-09 12:40:31,463::task::588::TaskManager.Task::(_updateState) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::moving from state preparing -> state finished
>Thread-1639::DEBUG::2012-12-09 12:40:31,463::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1639::DEBUG::2012-12-09 12:40:31,463::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1639::DEBUG::2012-12-09 12:40:31,464::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1639::DEBUG::2012-12-09 12:40:31,464::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1639::DEBUG::2012-12-09 12:40:31,465::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1639::DEBUG::2012-12-09 12:40:31,465::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1639::DEBUG::2012-12-09 12:40:31,465::task::978::TaskManager.Task::(_decref) Task=`63df352f-e89d-4c6a-92b9-9ba2ece3b88a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:32,715::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:32,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219008 s, 46.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:34,467::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:34,468::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:34,569::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:34,571::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:34,571::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:34,571::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:34,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:34,811::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152533 s, 67.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:36,818::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:36,856::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015552 s, 65.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:38,863::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:38,911::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237017 s, 43.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:39,747::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:39,759::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000822626 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:40,917::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:40,969::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268514 s, 38.1 MB/s\n'; <rc> = 0
>Thread-1646::DEBUG::2012-12-09 12:40:41,557::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1646::DEBUG::2012-12-09 12:40:41,558::task::588::TaskManager.Task::(_updateState) Task=`08fdfc3a-9adf-487c-9253-27b8eb196d34`::moving from state init -> state preparing
>Thread-1646::INFO::2012-12-09 12:40:41,559::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1646::INFO::2012-12-09 12:40:41,559::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1646::DEBUG::2012-12-09 12:40:41,560::task::1172::TaskManager.Task::(prepare) Task=`08fdfc3a-9adf-487c-9253-27b8eb196d34`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1646::DEBUG::2012-12-09 12:40:41,560::task::588::TaskManager.Task::(_updateState) Task=`08fdfc3a-9adf-487c-9253-27b8eb196d34`::moving from state preparing -> state finished
>Thread-1646::DEBUG::2012-12-09 12:40:41,560::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1646::DEBUG::2012-12-09 12:40:41,561::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1646::DEBUG::2012-12-09 12:40:41,561::task::978::TaskManager.Task::(_decref) Task=`08fdfc3a-9adf-487c-9253-27b8eb196d34`::ref 0 aborting False
>Thread-1647::DEBUG::2012-12-09 12:40:41,580::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1647::DEBUG::2012-12-09 12:40:41,581::task::588::TaskManager.Task::(_updateState) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::moving from state init -> state preparing
>Thread-1647::INFO::2012-12-09 12:40:41,581::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1647::DEBUG::2012-12-09 12:40:41,582::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f8b1967b-e8c4-4a9c-b2b4-156be723ec0f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1647::DEBUG::2012-12-09 12:40:41,582::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1647::DEBUG::2012-12-09 12:40:41,583::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1647::DEBUG::2012-12-09 12:40:41,583::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f8b1967b-e8c4-4a9c-b2b4-156be723ec0f`::Granted request
>Thread-1647::DEBUG::2012-12-09 12:40:41,584::task::817::TaskManager.Task::(resourceAcquired) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1647::DEBUG::2012-12-09 12:40:41,585::task::978::TaskManager.Task::(_decref) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::ref 1 aborting False
>Thread-1647::DEBUG::2012-12-09 12:40:41,586::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1647::DEBUG::2012-12-09 12:40:41,586::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1648::DEBUG::2012-12-09 12:40:41,639::task::588::TaskManager.Task::(_updateState) Task=`75b1c816-4653-426b-b437-22e63266ca12`::moving from state init -> state preparing
>Thread-1648::INFO::2012-12-09 12:40:41,640::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1648::INFO::2012-12-09 12:40:41,640::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130999088287', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-1648::DEBUG::2012-12-09 12:40:41,641::task::1172::TaskManager.Task::(prepare) Task=`75b1c816-4653-426b-b437-22e63266ca12`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130999088287', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-1648::DEBUG::2012-12-09 12:40:41,641::task::588::TaskManager.Task::(_updateState) Task=`75b1c816-4653-426b-b437-22e63266ca12`::moving from state preparing -> state finished
>Thread-1648::DEBUG::2012-12-09 12:40:41,641::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1648::DEBUG::2012-12-09 12:40:41,642::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1648::DEBUG::2012-12-09 12:40:41,642::task::978::TaskManager.Task::(_decref) Task=`75b1c816-4653-426b-b437-22e63266ca12`::ref 0 aborting False
>Thread-1647::DEBUG::2012-12-09 12:40:41,707::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1647::WARNING::2012-12-09 12:40:41,709::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1647::DEBUG::2012-12-09 12:40:41,710::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1647::WARNING::2012-12-09 12:40:41,711::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1647::INFO::2012-12-09 12:40:41,712::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1647::DEBUG::2012-12-09 12:40:41,712::task::1172::TaskManager.Task::(prepare) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1647::DEBUG::2012-12-09 12:40:41,713::task::588::TaskManager.Task::(_updateState) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::moving from state preparing -> state finished
>Thread-1647::DEBUG::2012-12-09 12:40:41,713::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1647::DEBUG::2012-12-09 12:40:41,713::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1647::DEBUG::2012-12-09 12:40:41,714::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1647::DEBUG::2012-12-09 12:40:41,714::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1647::DEBUG::2012-12-09 12:40:41,715::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1647::DEBUG::2012-12-09 12:40:41,715::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1647::DEBUG::2012-12-09 12:40:41,715::task::978::TaskManager.Task::(_decref) Task=`957f31cd-80a1-4621-94b3-370f17d6c7c5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:42,976::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:43,023::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214085 s, 47.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:44,573::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:44,574::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:44,684::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:44,685::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:44,686::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:44,686::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:45,030::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:45,079::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249525 s, 41.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:47,086::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:47,135::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220145 s, 46.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:49,142::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:49,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240488 s, 42.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:49,765::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:49,779::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000826585 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:51,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:51,249::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276203 s, 37.1 MB/s\n'; <rc> = 0
>Thread-1654::DEBUG::2012-12-09 12:40:51,831::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1654::DEBUG::2012-12-09 12:40:51,832::task::588::TaskManager.Task::(_updateState) Task=`4873f669-dc14-444b-a683-98e98d4d6c4f`::moving from state init -> state preparing
>Thread-1654::INFO::2012-12-09 12:40:51,833::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1654::INFO::2012-12-09 12:40:51,833::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1654::DEBUG::2012-12-09 12:40:51,834::task::1172::TaskManager.Task::(prepare) Task=`4873f669-dc14-444b-a683-98e98d4d6c4f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1654::DEBUG::2012-12-09 12:40:51,834::task::588::TaskManager.Task::(_updateState) Task=`4873f669-dc14-444b-a683-98e98d4d6c4f`::moving from state preparing -> state finished
>Thread-1654::DEBUG::2012-12-09 12:40:51,835::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1654::DEBUG::2012-12-09 12:40:51,835::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1654::DEBUG::2012-12-09 12:40:51,835::task::978::TaskManager.Task::(_decref) Task=`4873f669-dc14-444b-a683-98e98d4d6c4f`::ref 0 aborting False
>Thread-1655::DEBUG::2012-12-09 12:40:51,851::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1655::DEBUG::2012-12-09 12:40:51,852::task::588::TaskManager.Task::(_updateState) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::moving from state init -> state preparing
>Thread-1655::INFO::2012-12-09 12:40:51,852::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1655::DEBUG::2012-12-09 12:40:51,853::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a786ac1c-4207-438c-ba0f-f7b47ffa0861`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1655::DEBUG::2012-12-09 12:40:51,853::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1655::DEBUG::2012-12-09 12:40:51,853::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1655::DEBUG::2012-12-09 12:40:51,854::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a786ac1c-4207-438c-ba0f-f7b47ffa0861`::Granted request
>Thread-1655::DEBUG::2012-12-09 12:40:51,854::task::817::TaskManager.Task::(resourceAcquired) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1655::DEBUG::2012-12-09 12:40:51,855::task::978::TaskManager.Task::(_decref) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::ref 1 aborting False
>Thread-1655::DEBUG::2012-12-09 12:40:51,856::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1655::DEBUG::2012-12-09 12:40:51,856::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1655::DEBUG::2012-12-09 12:40:51,954::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1655::WARNING::2012-12-09 12:40:51,955::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1655::DEBUG::2012-12-09 12:40:51,956::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1655::WARNING::2012-12-09 12:40:51,956::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1655::INFO::2012-12-09 12:40:51,956::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1655::DEBUG::2012-12-09 12:40:51,957::task::1172::TaskManager.Task::(prepare) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1656::DEBUG::2012-12-09 12:40:51,959::task::588::TaskManager.Task::(_updateState) Task=`bb150498-4bd8-462d-9b7b-3957d47fd692`::moving from state init -> state preparing
>Thread-1655::DEBUG::2012-12-09 12:40:51,959::task::588::TaskManager.Task::(_updateState) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::moving from state preparing -> state finished
>Thread-1656::INFO::2012-12-09 12:40:51,959::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1655::DEBUG::2012-12-09 12:40:51,960::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1656::INFO::2012-12-09 12:40:51,961::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153670310974', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1655::DEBUG::2012-12-09 12:40:51,961::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1656::DEBUG::2012-12-09 12:40:51,961::task::1172::TaskManager.Task::(prepare) Task=`bb150498-4bd8-462d-9b7b-3957d47fd692`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153670310974', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1656::DEBUG::2012-12-09 12:40:51,962::task::588::TaskManager.Task::(_updateState) Task=`bb150498-4bd8-462d-9b7b-3957d47fd692`::moving from state preparing -> state finished
>Thread-1655::DEBUG::2012-12-09 12:40:51,962::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1656::DEBUG::2012-12-09 12:40:51,963::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1655::DEBUG::2012-12-09 12:40:51,964::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1656::DEBUG::2012-12-09 12:40:51,964::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1655::DEBUG::2012-12-09 12:40:51,964::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1656::DEBUG::2012-12-09 12:40:51,965::task::978::TaskManager.Task::(_decref) Task=`bb150498-4bd8-462d-9b7b-3957d47fd692`::ref 0 aborting False
>Thread-1655::DEBUG::2012-12-09 12:40:51,965::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1655::DEBUG::2012-12-09 12:40:51,969::task::978::TaskManager.Task::(_decref) Task=`293c0278-8cd3-4e58-92ff-8d924a7236e4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:40:53,255::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:53,302::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220376 s, 46.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:40:54,687::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:40:54,688::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:40:54,790::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:40:54,791::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:40:54,791::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:40:54,791::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:40:55,308::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:55,354::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211243 s, 48.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:57,361::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:57,405::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193375 s, 53.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:40:59,412::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:40:59,454::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185019 s, 55.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:40:59,785::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:40:59,800::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00079491 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:01,460::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:01,505::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202368 s, 50.6 MB/s\n'; <rc> = 0
>Thread-1662::DEBUG::2012-12-09 12:41:02,086::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1662::DEBUG::2012-12-09 12:41:02,087::task::588::TaskManager.Task::(_updateState) Task=`850c3ac9-b629-4a0f-8703-71d324f36774`::moving from state init -> state preparing
>Thread-1662::INFO::2012-12-09 12:41:02,088::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1662::INFO::2012-12-09 12:41:02,089::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1662::DEBUG::2012-12-09 12:41:02,089::task::1172::TaskManager.Task::(prepare) Task=`850c3ac9-b629-4a0f-8703-71d324f36774`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1662::DEBUG::2012-12-09 12:41:02,090::task::588::TaskManager.Task::(_updateState) Task=`850c3ac9-b629-4a0f-8703-71d324f36774`::moving from state preparing -> state finished
>Thread-1662::DEBUG::2012-12-09 12:41:02,090::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1662::DEBUG::2012-12-09 12:41:02,090::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1662::DEBUG::2012-12-09 12:41:02,091::task::978::TaskManager.Task::(_decref) Task=`850c3ac9-b629-4a0f-8703-71d324f36774`::ref 0 aborting False
>Thread-1663::DEBUG::2012-12-09 12:41:02,111::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1663::DEBUG::2012-12-09 12:41:02,111::task::588::TaskManager.Task::(_updateState) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::moving from state init -> state preparing
>Thread-1663::INFO::2012-12-09 12:41:02,112::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1663::DEBUG::2012-12-09 12:41:02,112::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`52ee13a8-d2a2-47cc-a431-d4608c07ebab`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1663::DEBUG::2012-12-09 12:41:02,112::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1663::DEBUG::2012-12-09 12:41:02,112::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1663::DEBUG::2012-12-09 12:41:02,113::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`52ee13a8-d2a2-47cc-a431-d4608c07ebab`::Granted request
>Thread-1663::DEBUG::2012-12-09 12:41:02,113::task::817::TaskManager.Task::(resourceAcquired) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1663::DEBUG::2012-12-09 12:41:02,113::task::978::TaskManager.Task::(_decref) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::ref 1 aborting False
>Thread-1663::DEBUG::2012-12-09 12:41:02,114::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1663::DEBUG::2012-12-09 12:41:02,114::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1664::DEBUG::2012-12-09 12:41:02,176::task::588::TaskManager.Task::(_updateState) Task=`a829f300-d4e4-471d-8a20-236407680247`::moving from state init -> state preparing
>Thread-1664::INFO::2012-12-09 12:41:02,176::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1664::INFO::2012-12-09 12:41:02,177::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0159201622009', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-1664::DEBUG::2012-12-09 12:41:02,177::task::1172::TaskManager.Task::(prepare) Task=`a829f300-d4e4-471d-8a20-236407680247`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0159201622009', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-1664::DEBUG::2012-12-09 12:41:02,177::task::588::TaskManager.Task::(_updateState) Task=`a829f300-d4e4-471d-8a20-236407680247`::moving from state preparing -> state finished
>Thread-1664::DEBUG::2012-12-09 12:41:02,177::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1664::DEBUG::2012-12-09 12:41:02,177::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1664::DEBUG::2012-12-09 12:41:02,178::task::978::TaskManager.Task::(_decref) Task=`a829f300-d4e4-471d-8a20-236407680247`::ref 0 aborting False
>Thread-1663::DEBUG::2012-12-09 12:41:02,227::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1663::WARNING::2012-12-09 12:41:02,228::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1663::DEBUG::2012-12-09 12:41:02,229::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1663::WARNING::2012-12-09 12:41:02,230::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1663::INFO::2012-12-09 12:41:02,231::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1663::DEBUG::2012-12-09 12:41:02,232::task::1172::TaskManager.Task::(prepare) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1663::DEBUG::2012-12-09 12:41:02,232::task::588::TaskManager.Task::(_updateState) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::moving from state preparing -> state finished
>Thread-1663::DEBUG::2012-12-09 12:41:02,233::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1663::DEBUG::2012-12-09 12:41:02,234::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1663::DEBUG::2012-12-09 12:41:02,234::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1663::DEBUG::2012-12-09 12:41:02,235::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1663::DEBUG::2012-12-09 12:41:02,235::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1663::DEBUG::2012-12-09 12:41:02,236::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1663::DEBUG::2012-12-09 12:41:02,236::task::978::TaskManager.Task::(_decref) Task=`aa09d91f-0193-480b-ad48-bb0da184cc99`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:03,512::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:03,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186447 s, 54.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:04,792::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:04,794::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:04,906::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:04,907::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:04,907::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:04,908::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:05,561::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:05,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018613 s, 55.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:07,610::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:07,665::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303724 s, 33.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:09,671::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:09,714::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199116 s, 51.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:09,806::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:09,818::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000914999 s, 4.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:11,721::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:11,769::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186864 s, 54.8 MB/s\n'; <rc> = 0
>Thread-1670::DEBUG::2012-12-09 12:41:12,300::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1670::DEBUG::2012-12-09 12:41:12,301::task::588::TaskManager.Task::(_updateState) Task=`506a5cc6-cc73-4f9c-ace3-79b330c60e2b`::moving from state init -> state preparing
>Thread-1670::INFO::2012-12-09 12:41:12,301::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1670::INFO::2012-12-09 12:41:12,301::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1670::DEBUG::2012-12-09 12:41:12,302::task::1172::TaskManager.Task::(prepare) Task=`506a5cc6-cc73-4f9c-ace3-79b330c60e2b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1670::DEBUG::2012-12-09 12:41:12,302::task::588::TaskManager.Task::(_updateState) Task=`506a5cc6-cc73-4f9c-ace3-79b330c60e2b`::moving from state preparing -> state finished
>Thread-1670::DEBUG::2012-12-09 12:41:12,303::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1670::DEBUG::2012-12-09 12:41:12,303::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1670::DEBUG::2012-12-09 12:41:12,304::task::978::TaskManager.Task::(_decref) Task=`506a5cc6-cc73-4f9c-ace3-79b330c60e2b`::ref 0 aborting False
>Thread-1671::DEBUG::2012-12-09 12:41:12,327::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1671::DEBUG::2012-12-09 12:41:12,328::task::588::TaskManager.Task::(_updateState) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::moving from state init -> state preparing
>Thread-1671::INFO::2012-12-09 12:41:12,329::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1671::DEBUG::2012-12-09 12:41:12,329::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c465864c-c498-4d7e-a430-6b609521c3b2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1671::DEBUG::2012-12-09 12:41:12,330::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1671::DEBUG::2012-12-09 12:41:12,330::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1671::DEBUG::2012-12-09 12:41:12,331::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c465864c-c498-4d7e-a430-6b609521c3b2`::Granted request
>Thread-1671::DEBUG::2012-12-09 12:41:12,331::task::817::TaskManager.Task::(resourceAcquired) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1671::DEBUG::2012-12-09 12:41:12,333::task::978::TaskManager.Task::(_decref) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::ref 1 aborting False
>Thread-1671::DEBUG::2012-12-09 12:41:12,335::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1671::DEBUG::2012-12-09 12:41:12,335::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1672::DEBUG::2012-12-09 12:41:12,401::task::588::TaskManager.Task::(_updateState) Task=`237942f6-aa58-47a6-ac62-d9cbfa8ac1ac`::moving from state init -> state preparing
>Thread-1672::INFO::2012-12-09 12:41:12,402::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1672::INFO::2012-12-09 12:41:12,402::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134389400482', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.5', 'code': 358, 'valid': False}}
>Thread-1672::DEBUG::2012-12-09 12:41:12,402::task::1172::TaskManager.Task::(prepare) Task=`237942f6-aa58-47a6-ac62-d9cbfa8ac1ac`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134389400482', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.5', 'code': 358, 'valid': False}}
>Thread-1672::DEBUG::2012-12-09 12:41:12,402::task::588::TaskManager.Task::(_updateState) Task=`237942f6-aa58-47a6-ac62-d9cbfa8ac1ac`::moving from state preparing -> state finished
>Thread-1672::DEBUG::2012-12-09 12:41:12,402::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1672::DEBUG::2012-12-09 12:41:12,403::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1672::DEBUG::2012-12-09 12:41:12,403::task::978::TaskManager.Task::(_decref) Task=`237942f6-aa58-47a6-ac62-d9cbfa8ac1ac`::ref 0 aborting False
>Thread-1671::DEBUG::2012-12-09 12:41:12,447::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1671::WARNING::2012-12-09 12:41:12,448::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1671::DEBUG::2012-12-09 12:41:12,449::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1671::WARNING::2012-12-09 12:41:12,449::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1671::INFO::2012-12-09 12:41:12,450::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1671::DEBUG::2012-12-09 12:41:12,450::task::1172::TaskManager.Task::(prepare) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1671::DEBUG::2012-12-09 12:41:12,451::task::588::TaskManager.Task::(_updateState) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::moving from state preparing -> state finished
>Thread-1671::DEBUG::2012-12-09 12:41:12,451::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1671::DEBUG::2012-12-09 12:41:12,451::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1671::DEBUG::2012-12-09 12:41:12,452::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1671::DEBUG::2012-12-09 12:41:12,452::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1671::DEBUG::2012-12-09 12:41:12,452::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1671::DEBUG::2012-12-09 12:41:12,453::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1671::DEBUG::2012-12-09 12:41:12,453::task::978::TaskManager.Task::(_decref) Task=`04698845-57c4-4afb-b731-a9aaedb4732c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:13,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:13,822::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221877 s, 46.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:14,909::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:14,910::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:15,020::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:15,022::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:15,022::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:15,022::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:15,829::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:15,875::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205303 s, 49.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:17,881::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:17,925::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193689 s, 52.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:19,824::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:19,837::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00048952 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:19,932::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:19,980::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244201 s, 41.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:21,987::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:22,033::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176871 s, 57.9 MB/s\n'; <rc> = 0
>Thread-1678::DEBUG::2012-12-09 12:41:23,043::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1678::DEBUG::2012-12-09 12:41:23,044::task::588::TaskManager.Task::(_updateState) Task=`06566b1b-451b-4307-8e4b-1c6d1d8bdcd9`::moving from state init -> state preparing
>Thread-1678::INFO::2012-12-09 12:41:23,044::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1678::INFO::2012-12-09 12:41:23,044::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1678::DEBUG::2012-12-09 12:41:23,044::task::1172::TaskManager.Task::(prepare) Task=`06566b1b-451b-4307-8e4b-1c6d1d8bdcd9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1678::DEBUG::2012-12-09 12:41:23,045::task::588::TaskManager.Task::(_updateState) Task=`06566b1b-451b-4307-8e4b-1c6d1d8bdcd9`::moving from state preparing -> state finished
>Thread-1678::DEBUG::2012-12-09 12:41:23,045::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1678::DEBUG::2012-12-09 12:41:23,045::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1678::DEBUG::2012-12-09 12:41:23,045::task::978::TaskManager.Task::(_decref) Task=`06566b1b-451b-4307-8e4b-1c6d1d8bdcd9`::ref 0 aborting False
>Thread-1679::DEBUG::2012-12-09 12:41:23,058::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1679::DEBUG::2012-12-09 12:41:23,058::task::588::TaskManager.Task::(_updateState) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::moving from state init -> state preparing
>Thread-1679::INFO::2012-12-09 12:41:23,059::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1679::DEBUG::2012-12-09 12:41:23,059::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`224823db-6a1a-4013-8c52-4967f49a1ef2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1679::DEBUG::2012-12-09 12:41:23,060::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1679::DEBUG::2012-12-09 12:41:23,060::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1679::DEBUG::2012-12-09 12:41:23,060::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`224823db-6a1a-4013-8c52-4967f49a1ef2`::Granted request
>Thread-1679::DEBUG::2012-12-09 12:41:23,061::task::817::TaskManager.Task::(resourceAcquired) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1679::DEBUG::2012-12-09 12:41:23,061::task::978::TaskManager.Task::(_decref) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::ref 1 aborting False
>Thread-1679::DEBUG::2012-12-09 12:41:23,062::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1679::DEBUG::2012-12-09 12:41:23,062::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1680::DEBUG::2012-12-09 12:41:23,134::task::588::TaskManager.Task::(_updateState) Task=`f5561134-5a17-44f6-8905-6cb0c399ed84`::moving from state init -> state preparing
>Thread-1680::INFO::2012-12-09 12:41:23,135::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1680::INFO::2012-12-09 12:41:23,135::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013396024704', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-1680::DEBUG::2012-12-09 12:41:23,135::task::1172::TaskManager.Task::(prepare) Task=`f5561134-5a17-44f6-8905-6cb0c399ed84`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013396024704', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-1680::DEBUG::2012-12-09 12:41:23,136::task::588::TaskManager.Task::(_updateState) Task=`f5561134-5a17-44f6-8905-6cb0c399ed84`::moving from state preparing -> state finished
>Thread-1680::DEBUG::2012-12-09 12:41:23,136::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1680::DEBUG::2012-12-09 12:41:23,136::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1680::DEBUG::2012-12-09 12:41:23,136::task::978::TaskManager.Task::(_decref) Task=`f5561134-5a17-44f6-8905-6cb0c399ed84`::ref 0 aborting False
>Thread-1679::DEBUG::2012-12-09 12:41:23,164::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1679::WARNING::2012-12-09 12:41:23,165::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1679::DEBUG::2012-12-09 12:41:23,166::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1679::WARNING::2012-12-09 12:41:23,166::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1679::INFO::2012-12-09 12:41:23,167::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1679::DEBUG::2012-12-09 12:41:23,167::task::1172::TaskManager.Task::(prepare) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1679::DEBUG::2012-12-09 12:41:23,167::task::588::TaskManager.Task::(_updateState) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::moving from state preparing -> state finished
>Thread-1679::DEBUG::2012-12-09 12:41:23,167::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1679::DEBUG::2012-12-09 12:41:23,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1679::DEBUG::2012-12-09 12:41:23,168::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1679::DEBUG::2012-12-09 12:41:23,168::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1679::DEBUG::2012-12-09 12:41:23,168::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1679::DEBUG::2012-12-09 12:41:23,168::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1679::DEBUG::2012-12-09 12:41:23,169::task::978::TaskManager.Task::(_decref) Task=`fb072090-6988-4f2a-a93f-01fe13fd8b65`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:24,040::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:24,085::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196547 s, 52.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:25,024::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:25,025::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:25,145::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:25,146::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:25,147::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:25,147::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:26,092::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:26,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210438 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:28,143::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:28,191::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239094 s, 42.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:29,840::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:29,848::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000524348 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:30,197::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:30,241::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181654 s, 56.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:32,247::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:32,295::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021947 s, 46.7 MB/s\n'; <rc> = 0
>Thread-1686::DEBUG::2012-12-09 12:41:33,340::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1686::DEBUG::2012-12-09 12:41:33,341::task::588::TaskManager.Task::(_updateState) Task=`6255fd46-7874-4b96-9770-aea15cda8521`::moving from state init -> state preparing
>Thread-1686::INFO::2012-12-09 12:41:33,342::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1686::INFO::2012-12-09 12:41:33,343::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1686::DEBUG::2012-12-09 12:41:33,343::task::1172::TaskManager.Task::(prepare) Task=`6255fd46-7874-4b96-9770-aea15cda8521`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1686::DEBUG::2012-12-09 12:41:33,343::task::588::TaskManager.Task::(_updateState) Task=`6255fd46-7874-4b96-9770-aea15cda8521`::moving from state preparing -> state finished
>Thread-1686::DEBUG::2012-12-09 12:41:33,344::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1686::DEBUG::2012-12-09 12:41:33,344::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1686::DEBUG::2012-12-09 12:41:33,345::task::978::TaskManager.Task::(_decref) Task=`6255fd46-7874-4b96-9770-aea15cda8521`::ref 0 aborting False
>Thread-1687::DEBUG::2012-12-09 12:41:33,368::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1687::DEBUG::2012-12-09 12:41:33,368::task::588::TaskManager.Task::(_updateState) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::moving from state init -> state preparing
>Thread-1687::INFO::2012-12-09 12:41:33,369::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1687::DEBUG::2012-12-09 12:41:33,370::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2cd2e4f6-74d9-4b73-a7d4-902f1cba7d62`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1687::DEBUG::2012-12-09 12:41:33,370::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1687::DEBUG::2012-12-09 12:41:33,371::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1687::DEBUG::2012-12-09 12:41:33,371::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2cd2e4f6-74d9-4b73-a7d4-902f1cba7d62`::Granted request
>Thread-1687::DEBUG::2012-12-09 12:41:33,372::task::817::TaskManager.Task::(resourceAcquired) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1687::DEBUG::2012-12-09 12:41:33,372::task::978::TaskManager.Task::(_decref) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::ref 1 aborting False
>Thread-1687::DEBUG::2012-12-09 12:41:33,374::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1687::DEBUG::2012-12-09 12:41:33,375::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1688::DEBUG::2012-12-09 12:41:33,457::task::588::TaskManager.Task::(_updateState) Task=`2b952255-c262-48a4-9110-eeffc1e90eb7`::moving from state init -> state preparing
>Thread-1688::INFO::2012-12-09 12:41:33,457::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1688::INFO::2012-12-09 12:41:33,458::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00830578804016', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.3', 'code': 358, 'valid': False}}
>Thread-1688::DEBUG::2012-12-09 12:41:33,458::task::1172::TaskManager.Task::(prepare) Task=`2b952255-c262-48a4-9110-eeffc1e90eb7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00830578804016', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.3', 'code': 358, 'valid': False}}
>Thread-1688::DEBUG::2012-12-09 12:41:33,458::task::588::TaskManager.Task::(_updateState) Task=`2b952255-c262-48a4-9110-eeffc1e90eb7`::moving from state preparing -> state finished
>Thread-1688::DEBUG::2012-12-09 12:41:33,458::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1688::DEBUG::2012-12-09 12:41:33,458::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1688::DEBUG::2012-12-09 12:41:33,458::task::978::TaskManager.Task::(_decref) Task=`2b952255-c262-48a4-9110-eeffc1e90eb7`::ref 0 aborting False
>Thread-1687::DEBUG::2012-12-09 12:41:33,474::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1687::WARNING::2012-12-09 12:41:33,477::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1687::DEBUG::2012-12-09 12:41:33,477::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1687::WARNING::2012-12-09 12:41:33,478::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1687::INFO::2012-12-09 12:41:33,478::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1687::DEBUG::2012-12-09 12:41:33,478::task::1172::TaskManager.Task::(prepare) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1687::DEBUG::2012-12-09 12:41:33,479::task::588::TaskManager.Task::(_updateState) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::moving from state preparing -> state finished
>Thread-1687::DEBUG::2012-12-09 12:41:33,479::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1687::DEBUG::2012-12-09 12:41:33,479::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1687::DEBUG::2012-12-09 12:41:33,479::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1687::DEBUG::2012-12-09 12:41:33,480::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1687::DEBUG::2012-12-09 12:41:33,480::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1687::DEBUG::2012-12-09 12:41:33,480::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1687::DEBUG::2012-12-09 12:41:33,480::task::978::TaskManager.Task::(_decref) Task=`f8456b77-2c92-49e7-98fb-b09bf8a29767`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:34,301::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:34,350::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243531 s, 42.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:35,149::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:35,150::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:35,265::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:35,266::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:35,266::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:35,267::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:36,357::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:36,409::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02715 s, 37.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:38,415::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:38,454::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160998 s, 63.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:39,851::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:39,863::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000445909 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:40,460::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:40,504::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202648 s, 50.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:42,511::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:42,561::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249703 s, 41.0 MB/s\n'; <rc> = 0
>Thread-1694::DEBUG::2012-12-09 12:41:43,638::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1694::DEBUG::2012-12-09 12:41:43,639::task::588::TaskManager.Task::(_updateState) Task=`b1e3a1ca-7eab-4c67-a5fe-b1baf2cd2444`::moving from state init -> state preparing
>Thread-1694::INFO::2012-12-09 12:41:43,639::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1694::INFO::2012-12-09 12:41:43,640::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1694::DEBUG::2012-12-09 12:41:43,640::task::1172::TaskManager.Task::(prepare) Task=`b1e3a1ca-7eab-4c67-a5fe-b1baf2cd2444`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1694::DEBUG::2012-12-09 12:41:43,641::task::588::TaskManager.Task::(_updateState) Task=`b1e3a1ca-7eab-4c67-a5fe-b1baf2cd2444`::moving from state preparing -> state finished
>Thread-1694::DEBUG::2012-12-09 12:41:43,641::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1694::DEBUG::2012-12-09 12:41:43,642::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1694::DEBUG::2012-12-09 12:41:43,642::task::978::TaskManager.Task::(_decref) Task=`b1e3a1ca-7eab-4c67-a5fe-b1baf2cd2444`::ref 0 aborting False
>Thread-1695::DEBUG::2012-12-09 12:41:43,668::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1695::DEBUG::2012-12-09 12:41:43,668::task::588::TaskManager.Task::(_updateState) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::moving from state init -> state preparing
>Thread-1695::INFO::2012-12-09 12:41:43,669::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1695::DEBUG::2012-12-09 12:41:43,670::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67ce8f86-2e7e-4d5b-bd36-cf1de3eb4f5a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1695::DEBUG::2012-12-09 12:41:43,670::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1695::DEBUG::2012-12-09 12:41:43,671::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1695::DEBUG::2012-12-09 12:41:43,671::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67ce8f86-2e7e-4d5b-bd36-cf1de3eb4f5a`::Granted request
>Thread-1695::DEBUG::2012-12-09 12:41:43,672::task::817::TaskManager.Task::(resourceAcquired) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1695::DEBUG::2012-12-09 12:41:43,672::task::978::TaskManager.Task::(_decref) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::ref 1 aborting False
>Thread-1695::DEBUG::2012-12-09 12:41:43,674::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1695::DEBUG::2012-12-09 12:41:43,675::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1696::DEBUG::2012-12-09 12:41:43,737::task::588::TaskManager.Task::(_updateState) Task=`fc046098-f9f6-4317-acba-1add643012a1`::moving from state init -> state preparing
>Thread-1696::INFO::2012-12-09 12:41:43,738::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1696::INFO::2012-12-09 12:41:43,739::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118582248688', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-1696::DEBUG::2012-12-09 12:41:43,739::task::1172::TaskManager.Task::(prepare) Task=`fc046098-f9f6-4317-acba-1add643012a1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118582248688', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-1696::DEBUG::2012-12-09 12:41:43,739::task::588::TaskManager.Task::(_updateState) Task=`fc046098-f9f6-4317-acba-1add643012a1`::moving from state preparing -> state finished
>Thread-1696::DEBUG::2012-12-09 12:41:43,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1696::DEBUG::2012-12-09 12:41:43,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1696::DEBUG::2012-12-09 12:41:43,740::task::978::TaskManager.Task::(_decref) Task=`fc046098-f9f6-4317-acba-1add643012a1`::ref 0 aborting False
>Thread-1695::DEBUG::2012-12-09 12:41:43,798::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1695::WARNING::2012-12-09 12:41:43,799::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1695::DEBUG::2012-12-09 12:41:43,799::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1695::WARNING::2012-12-09 12:41:43,799::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1695::INFO::2012-12-09 12:41:43,800::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1695::DEBUG::2012-12-09 12:41:43,800::task::1172::TaskManager.Task::(prepare) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1695::DEBUG::2012-12-09 12:41:43,800::task::588::TaskManager.Task::(_updateState) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::moving from state preparing -> state finished
>Thread-1695::DEBUG::2012-12-09 12:41:43,800::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1695::DEBUG::2012-12-09 12:41:43,800::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1695::DEBUG::2012-12-09 12:41:43,801::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1695::DEBUG::2012-12-09 12:41:43,801::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1695::DEBUG::2012-12-09 12:41:43,801::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1695::DEBUG::2012-12-09 12:41:43,801::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1695::DEBUG::2012-12-09 12:41:43,801::task::978::TaskManager.Task::(_decref) Task=`93a579e9-e5d7-4d57-9767-7f041908a85b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:44,567::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:44,621::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272982 s, 37.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:45,268::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:45,269::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:45,386::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:45,387::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:45,387::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:45,388::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:46,627::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:46,669::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162025 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:48,676::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:48,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227504 s, 45.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:49,866::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:49,881::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000967556 s, 4.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:50,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:50,780::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245844 s, 41.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:52,786::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:52,838::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270184 s, 37.9 MB/s\n'; <rc> = 0
>Thread-1702::DEBUG::2012-12-09 12:41:53,972::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1702::DEBUG::2012-12-09 12:41:53,973::task::588::TaskManager.Task::(_updateState) Task=`8aa0bab8-aa1a-44cd-b53b-ac305ff3394f`::moving from state init -> state preparing
>Thread-1702::INFO::2012-12-09 12:41:53,973::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1702::INFO::2012-12-09 12:41:53,973::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1702::DEBUG::2012-12-09 12:41:53,973::task::1172::TaskManager.Task::(prepare) Task=`8aa0bab8-aa1a-44cd-b53b-ac305ff3394f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1702::DEBUG::2012-12-09 12:41:53,974::task::588::TaskManager.Task::(_updateState) Task=`8aa0bab8-aa1a-44cd-b53b-ac305ff3394f`::moving from state preparing -> state finished
>Thread-1702::DEBUG::2012-12-09 12:41:53,974::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1702::DEBUG::2012-12-09 12:41:53,974::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1702::DEBUG::2012-12-09 12:41:53,974::task::978::TaskManager.Task::(_decref) Task=`8aa0bab8-aa1a-44cd-b53b-ac305ff3394f`::ref 0 aborting False
>Thread-1703::DEBUG::2012-12-09 12:41:53,993::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1703::DEBUG::2012-12-09 12:41:53,994::task::588::TaskManager.Task::(_updateState) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::moving from state init -> state preparing
>Thread-1703::INFO::2012-12-09 12:41:53,995::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1703::DEBUG::2012-12-09 12:41:53,996::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`933bb4c1-e426-4b23-adbd-01d87cb04e62`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1703::DEBUG::2012-12-09 12:41:53,996::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1703::DEBUG::2012-12-09 12:41:53,996::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1703::DEBUG::2012-12-09 12:41:53,997::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`933bb4c1-e426-4b23-adbd-01d87cb04e62`::Granted request
>Thread-1703::DEBUG::2012-12-09 12:41:53,997::task::817::TaskManager.Task::(resourceAcquired) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1703::DEBUG::2012-12-09 12:41:53,997::task::978::TaskManager.Task::(_decref) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::ref 1 aborting False
>Thread-1703::DEBUG::2012-12-09 12:41:53,998::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1703::DEBUG::2012-12-09 12:41:53,998::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1703::DEBUG::2012-12-09 12:41:54,101::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1703::WARNING::2012-12-09 12:41:54,102::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1703::DEBUG::2012-12-09 12:41:54,102::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1703::WARNING::2012-12-09 12:41:54,103::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1703::INFO::2012-12-09 12:41:54,103::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1703::DEBUG::2012-12-09 12:41:54,104::task::1172::TaskManager.Task::(prepare) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1703::DEBUG::2012-12-09 12:41:54,104::task::588::TaskManager.Task::(_updateState) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::moving from state preparing -> state finished
>Thread-1703::DEBUG::2012-12-09 12:41:54,105::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1703::DEBUG::2012-12-09 12:41:54,105::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1703::DEBUG::2012-12-09 12:41:54,106::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1703::DEBUG::2012-12-09 12:41:54,106::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1703::DEBUG::2012-12-09 12:41:54,106::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1703::DEBUG::2012-12-09 12:41:54,106::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1703::DEBUG::2012-12-09 12:41:54,107::task::978::TaskManager.Task::(_decref) Task=`75a48bf5-5a19-45e1-9c12-6af796321983`::ref 0 aborting False
>Thread-1704::DEBUG::2012-12-09 12:41:54,493::task::588::TaskManager.Task::(_updateState) Task=`9167f7ef-25ce-433f-abcd-4f1168df9344`::moving from state init -> state preparing
>Thread-1704::INFO::2012-12-09 12:41:54,493::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1704::INFO::2012-12-09 12:41:54,494::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016471862793', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-1704::DEBUG::2012-12-09 12:41:54,494::task::1172::TaskManager.Task::(prepare) Task=`9167f7ef-25ce-433f-abcd-4f1168df9344`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016471862793', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-1704::DEBUG::2012-12-09 12:41:54,494::task::588::TaskManager.Task::(_updateState) Task=`9167f7ef-25ce-433f-abcd-4f1168df9344`::moving from state preparing -> state finished
>Thread-1704::DEBUG::2012-12-09 12:41:54,494::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1704::DEBUG::2012-12-09 12:41:54,494::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1704::DEBUG::2012-12-09 12:41:54,495::task::978::TaskManager.Task::(_decref) Task=`9167f7ef-25ce-433f-abcd-4f1168df9344`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:41:54,844::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:54,882::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016131 s, 63.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:41:55,389::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:41:55,389::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:41:55,484::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:41:55,485::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:41:55,485::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:41:55,486::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:41:56,889::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:56,934::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203466 s, 50.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:41:58,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:41:58,984::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199467 s, 51.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:41:59,887::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:41:59,901::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000897903 s, 4.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:00,991::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:01,034::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184541 s, 55.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:03,040::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:03,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016474 s, 62.2 MB/s\n'; <rc> = 0
>Thread-1710::DEBUG::2012-12-09 12:42:04,180::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1710::DEBUG::2012-12-09 12:42:04,180::task::588::TaskManager.Task::(_updateState) Task=`6e98a3a0-fded-4b70-a05c-0053004f527b`::moving from state init -> state preparing
>Thread-1710::INFO::2012-12-09 12:42:04,181::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1710::INFO::2012-12-09 12:42:04,181::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1710::DEBUG::2012-12-09 12:42:04,181::task::1172::TaskManager.Task::(prepare) Task=`6e98a3a0-fded-4b70-a05c-0053004f527b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1710::DEBUG::2012-12-09 12:42:04,182::task::588::TaskManager.Task::(_updateState) Task=`6e98a3a0-fded-4b70-a05c-0053004f527b`::moving from state preparing -> state finished
>Thread-1710::DEBUG::2012-12-09 12:42:04,182::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1710::DEBUG::2012-12-09 12:42:04,182::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1710::DEBUG::2012-12-09 12:42:04,183::task::978::TaskManager.Task::(_decref) Task=`6e98a3a0-fded-4b70-a05c-0053004f527b`::ref 0 aborting False
>Thread-1711::DEBUG::2012-12-09 12:42:04,199::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1711::DEBUG::2012-12-09 12:42:04,200::task::588::TaskManager.Task::(_updateState) Task=`b7f0655b-1905-4253-b631-46199298b596`::moving from state init -> state preparing
>Thread-1711::INFO::2012-12-09 12:42:04,200::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1711::DEBUG::2012-12-09 12:42:04,201::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ddc42a33-813a-4b7b-845f-51e82f78fbde`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1711::DEBUG::2012-12-09 12:42:04,201::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1711::DEBUG::2012-12-09 12:42:04,202::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1711::DEBUG::2012-12-09 12:42:04,202::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ddc42a33-813a-4b7b-845f-51e82f78fbde`::Granted request
>Thread-1711::DEBUG::2012-12-09 12:42:04,203::task::817::TaskManager.Task::(resourceAcquired) Task=`b7f0655b-1905-4253-b631-46199298b596`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1711::DEBUG::2012-12-09 12:42:04,203::task::978::TaskManager.Task::(_decref) Task=`b7f0655b-1905-4253-b631-46199298b596`::ref 1 aborting False
>Thread-1711::DEBUG::2012-12-09 12:42:04,205::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1711::DEBUG::2012-12-09 12:42:04,206::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1711::DEBUG::2012-12-09 12:42:04,302::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1711::WARNING::2012-12-09 12:42:04,303::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1711::DEBUG::2012-12-09 12:42:04,303::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1711::WARNING::2012-12-09 12:42:04,304::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1711::INFO::2012-12-09 12:42:04,304::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1711::DEBUG::2012-12-09 12:42:04,305::task::1172::TaskManager.Task::(prepare) Task=`b7f0655b-1905-4253-b631-46199298b596`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1711::DEBUG::2012-12-09 12:42:04,305::task::588::TaskManager.Task::(_updateState) Task=`b7f0655b-1905-4253-b631-46199298b596`::moving from state preparing -> state finished
>Thread-1711::DEBUG::2012-12-09 12:42:04,305::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1711::DEBUG::2012-12-09 12:42:04,305::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1711::DEBUG::2012-12-09 12:42:04,306::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1711::DEBUG::2012-12-09 12:42:04,306::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1711::DEBUG::2012-12-09 12:42:04,306::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1711::DEBUG::2012-12-09 12:42:04,306::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1711::DEBUG::2012-12-09 12:42:04,306::task::978::TaskManager.Task::(_decref) Task=`b7f0655b-1905-4253-b631-46199298b596`::ref 0 aborting False
>Thread-1712::DEBUG::2012-12-09 12:42:04,818::task::588::TaskManager.Task::(_updateState) Task=`8fd2e3c0-0dd8-4e88-b51c-b277baed43fe`::moving from state init -> state preparing
>Thread-1712::INFO::2012-12-09 12:42:04,819::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1712::INFO::2012-12-09 12:42:04,819::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152771472931', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-1712::DEBUG::2012-12-09 12:42:04,820::task::1172::TaskManager.Task::(prepare) Task=`8fd2e3c0-0dd8-4e88-b51c-b277baed43fe`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152771472931', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-1712::DEBUG::2012-12-09 12:42:04,820::task::588::TaskManager.Task::(_updateState) Task=`8fd2e3c0-0dd8-4e88-b51c-b277baed43fe`::moving from state preparing -> state finished
>Thread-1712::DEBUG::2012-12-09 12:42:04,821::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1712::DEBUG::2012-12-09 12:42:04,821::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1712::DEBUG::2012-12-09 12:42:04,821::task::978::TaskManager.Task::(_decref) Task=`8fd2e3c0-0dd8-4e88-b51c-b277baed43fe`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:42:05,087::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:05,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272327 s, 37.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:42:05,487::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:05,488::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:42:05,601::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:05,603::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:05,603::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:05,603::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:07,146::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:07,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276264 s, 37.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:09,207::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:09,253::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209356 s, 48.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:42:09,907::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:09,919::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000484851 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:11,259::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:11,301::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01837 s, 55.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:13,308::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:13,346::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165639 s, 61.8 MB/s\n'; <rc> = 0
>Thread-1718::DEBUG::2012-12-09 12:42:14,514::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1718::DEBUG::2012-12-09 12:42:14,515::task::588::TaskManager.Task::(_updateState) Task=`fec328b9-048b-4b41-8ace-994a7f4089f0`::moving from state init -> state preparing
>Thread-1718::INFO::2012-12-09 12:42:14,515::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1718::INFO::2012-12-09 12:42:14,515::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1718::DEBUG::2012-12-09 12:42:14,516::task::1172::TaskManager.Task::(prepare) Task=`fec328b9-048b-4b41-8ace-994a7f4089f0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1718::DEBUG::2012-12-09 12:42:14,516::task::588::TaskManager.Task::(_updateState) Task=`fec328b9-048b-4b41-8ace-994a7f4089f0`::moving from state preparing -> state finished
>Thread-1718::DEBUG::2012-12-09 12:42:14,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1718::DEBUG::2012-12-09 12:42:14,516::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1718::DEBUG::2012-12-09 12:42:14,517::task::978::TaskManager.Task::(_decref) Task=`fec328b9-048b-4b41-8ace-994a7f4089f0`::ref 0 aborting False
>Thread-1719::DEBUG::2012-12-09 12:42:14,529::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1719::DEBUG::2012-12-09 12:42:14,530::task::588::TaskManager.Task::(_updateState) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::moving from state init -> state preparing
>Thread-1719::INFO::2012-12-09 12:42:14,530::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1719::DEBUG::2012-12-09 12:42:14,530::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4dd4a827-8644-4c50-80c5-b280f895490d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1719::DEBUG::2012-12-09 12:42:14,530::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1719::DEBUG::2012-12-09 12:42:14,531::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1719::DEBUG::2012-12-09 12:42:14,531::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4dd4a827-8644-4c50-80c5-b280f895490d`::Granted request
>Thread-1719::DEBUG::2012-12-09 12:42:14,531::task::817::TaskManager.Task::(resourceAcquired) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1719::DEBUG::2012-12-09 12:42:14,531::task::978::TaskManager.Task::(_decref) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::ref 1 aborting False
>Thread-1719::DEBUG::2012-12-09 12:42:14,532::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1719::DEBUG::2012-12-09 12:42:14,532::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1719::DEBUG::2012-12-09 12:42:14,630::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1719::WARNING::2012-12-09 12:42:14,631::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1719::DEBUG::2012-12-09 12:42:14,631::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1719::WARNING::2012-12-09 12:42:14,632::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1719::INFO::2012-12-09 12:42:14,632::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1719::DEBUG::2012-12-09 12:42:14,632::task::1172::TaskManager.Task::(prepare) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1719::DEBUG::2012-12-09 12:42:14,633::task::588::TaskManager.Task::(_updateState) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::moving from state preparing -> state finished
>Thread-1719::DEBUG::2012-12-09 12:42:14,633::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1719::DEBUG::2012-12-09 12:42:14,633::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1719::DEBUG::2012-12-09 12:42:14,633::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1719::DEBUG::2012-12-09 12:42:14,634::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1719::DEBUG::2012-12-09 12:42:14,634::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1719::DEBUG::2012-12-09 12:42:14,634::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1719::DEBUG::2012-12-09 12:42:14,634::task::978::TaskManager.Task::(_decref) Task=`d9be6801-82ae-4000-90f6-cb3ea6559dcb`::ref 0 aborting False
>Thread-1720::DEBUG::2012-12-09 12:42:15,069::task::588::TaskManager.Task::(_updateState) Task=`8ea44ed9-a227-4ca0-b5be-8250ee34a5fe`::moving from state init -> state preparing
>Thread-1720::INFO::2012-12-09 12:42:15,069::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1720::INFO::2012-12-09 12:42:15,070::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013011932373', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-1720::DEBUG::2012-12-09 12:42:15,070::task::1172::TaskManager.Task::(prepare) Task=`8ea44ed9-a227-4ca0-b5be-8250ee34a5fe`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.013011932373', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.5', 'code': 358, 'valid': False}}
>Thread-1720::DEBUG::2012-12-09 12:42:15,070::task::588::TaskManager.Task::(_updateState) Task=`8ea44ed9-a227-4ca0-b5be-8250ee34a5fe`::moving from state preparing -> state finished
>Thread-1720::DEBUG::2012-12-09 12:42:15,071::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1720::DEBUG::2012-12-09 12:42:15,071::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1720::DEBUG::2012-12-09 12:42:15,071::task::978::TaskManager.Task::(_decref) Task=`8ea44ed9-a227-4ca0-b5be-8250ee34a5fe`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:42:15,353::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:15,392::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160695 s, 63.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:42:15,604::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:15,605::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:42:15,702::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:15,703::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:15,703::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:15,703::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:17,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:17,441::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165594 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:19,447::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:19,488::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165731 s, 61.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:42:19,922::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:19,937::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000516533 s, 7.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:21,494::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:21,536::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197795 s, 51.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:23,543::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:23,581::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162034 s, 63.2 MB/s\n'; <rc> = 0
>Thread-1726::DEBUG::2012-12-09 12:42:24,810::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1726::DEBUG::2012-12-09 12:42:24,811::task::588::TaskManager.Task::(_updateState) Task=`fb988d57-3889-4764-b727-12d310bc2bbb`::moving from state init -> state preparing
>Thread-1726::INFO::2012-12-09 12:42:24,811::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1726::INFO::2012-12-09 12:42:24,812::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1726::DEBUG::2012-12-09 12:42:24,812::task::1172::TaskManager.Task::(prepare) Task=`fb988d57-3889-4764-b727-12d310bc2bbb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1726::DEBUG::2012-12-09 12:42:24,812::task::588::TaskManager.Task::(_updateState) Task=`fb988d57-3889-4764-b727-12d310bc2bbb`::moving from state preparing -> state finished
>Thread-1726::DEBUG::2012-12-09 12:42:24,812::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1726::DEBUG::2012-12-09 12:42:24,812::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1726::DEBUG::2012-12-09 12:42:24,812::task::978::TaskManager.Task::(_decref) Task=`fb988d57-3889-4764-b727-12d310bc2bbb`::ref 0 aborting False
>Thread-1727::DEBUG::2012-12-09 12:42:24,825::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1727::DEBUG::2012-12-09 12:42:24,826::task::588::TaskManager.Task::(_updateState) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::moving from state init -> state preparing
>Thread-1727::INFO::2012-12-09 12:42:24,826::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1727::DEBUG::2012-12-09 12:42:24,826::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5aecdbe4-99ce-4158-8159-0052f7cf39ed`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1727::DEBUG::2012-12-09 12:42:24,826::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1727::DEBUG::2012-12-09 12:42:24,827::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1727::DEBUG::2012-12-09 12:42:24,827::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5aecdbe4-99ce-4158-8159-0052f7cf39ed`::Granted request
>Thread-1727::DEBUG::2012-12-09 12:42:24,828::task::817::TaskManager.Task::(resourceAcquired) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1727::DEBUG::2012-12-09 12:42:24,828::task::978::TaskManager.Task::(_decref) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::ref 1 aborting False
>Thread-1727::DEBUG::2012-12-09 12:42:24,829::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1727::DEBUG::2012-12-09 12:42:24,829::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1727::DEBUG::2012-12-09 12:42:24,925::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1727::WARNING::2012-12-09 12:42:24,927::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1727::DEBUG::2012-12-09 12:42:24,927::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1727::WARNING::2012-12-09 12:42:24,927::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1727::INFO::2012-12-09 12:42:24,928::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1727::DEBUG::2012-12-09 12:42:24,928::task::1172::TaskManager.Task::(prepare) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1727::DEBUG::2012-12-09 12:42:24,928::task::588::TaskManager.Task::(_updateState) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::moving from state preparing -> state finished
>Thread-1727::DEBUG::2012-12-09 12:42:24,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1727::DEBUG::2012-12-09 12:42:24,929::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1727::DEBUG::2012-12-09 12:42:24,929::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1727::DEBUG::2012-12-09 12:42:24,929::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1727::DEBUG::2012-12-09 12:42:24,929::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1727::DEBUG::2012-12-09 12:42:24,929::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1727::DEBUG::2012-12-09 12:42:24,930::task::978::TaskManager.Task::(_decref) Task=`8f6e4047-199d-4a98-a82d-87ed505a2439`::ref 0 aborting False
>Thread-1728::DEBUG::2012-12-09 12:42:25,285::task::588::TaskManager.Task::(_updateState) Task=`e09fc3ea-a524-465a-a572-f7f2e4e39b85`::moving from state init -> state preparing
>Thread-1728::INFO::2012-12-09 12:42:25,285::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1728::INFO::2012-12-09 12:42:25,285::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149130821228', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-1728::DEBUG::2012-12-09 12:42:25,285::task::1172::TaskManager.Task::(prepare) Task=`e09fc3ea-a524-465a-a572-f7f2e4e39b85`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149130821228', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-1728::DEBUG::2012-12-09 12:42:25,286::task::588::TaskManager.Task::(_updateState) Task=`e09fc3ea-a524-465a-a572-f7f2e4e39b85`::moving from state preparing -> state finished
>Thread-1728::DEBUG::2012-12-09 12:42:25,286::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1728::DEBUG::2012-12-09 12:42:25,286::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1728::DEBUG::2012-12-09 12:42:25,286::task::978::TaskManager.Task::(_decref) Task=`e09fc3ea-a524-465a-a572-f7f2e4e39b85`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:42:25,588::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:25,635::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214044 s, 47.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:42:25,704::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:25,705::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:42:25,811::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:25,812::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:25,812::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:25,813::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:27,641::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:27,680::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161982 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:29,686::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:29,725::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162647 s, 63.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:42:29,939::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:29,947::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000483431 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:31,731::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:31,777::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238524 s, 42.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:33,784::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:33,825::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170596 s, 60.0 MB/s\n'; <rc> = 0
>Thread-1734::DEBUG::2012-12-09 12:42:35,053::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1734::DEBUG::2012-12-09 12:42:35,053::task::588::TaskManager.Task::(_updateState) Task=`54bf6fcd-7865-43f3-a0a3-15f72de9c3ac`::moving from state init -> state preparing
>Thread-1734::INFO::2012-12-09 12:42:35,054::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1734::INFO::2012-12-09 12:42:35,054::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1734::DEBUG::2012-12-09 12:42:35,054::task::1172::TaskManager.Task::(prepare) Task=`54bf6fcd-7865-43f3-a0a3-15f72de9c3ac`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1734::DEBUG::2012-12-09 12:42:35,054::task::588::TaskManager.Task::(_updateState) Task=`54bf6fcd-7865-43f3-a0a3-15f72de9c3ac`::moving from state preparing -> state finished
>Thread-1734::DEBUG::2012-12-09 12:42:35,055::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1734::DEBUG::2012-12-09 12:42:35,055::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1734::DEBUG::2012-12-09 12:42:35,055::task::978::TaskManager.Task::(_decref) Task=`54bf6fcd-7865-43f3-a0a3-15f72de9c3ac`::ref 0 aborting False
>Thread-1735::DEBUG::2012-12-09 12:42:35,067::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1735::DEBUG::2012-12-09 12:42:35,068::task::588::TaskManager.Task::(_updateState) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::moving from state init -> state preparing
>Thread-1735::INFO::2012-12-09 12:42:35,068::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1735::DEBUG::2012-12-09 12:42:35,068::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ee70513-b2a5-4b1f-8e07-53b7e888a1b8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1735::DEBUG::2012-12-09 12:42:35,068::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1735::DEBUG::2012-12-09 12:42:35,069::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1735::DEBUG::2012-12-09 12:42:35,069::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ee70513-b2a5-4b1f-8e07-53b7e888a1b8`::Granted request
>Thread-1735::DEBUG::2012-12-09 12:42:35,069::task::817::TaskManager.Task::(resourceAcquired) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1735::DEBUG::2012-12-09 12:42:35,069::task::978::TaskManager.Task::(_decref) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::ref 1 aborting False
>Thread-1735::DEBUG::2012-12-09 12:42:35,070::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1735::DEBUG::2012-12-09 12:42:35,071::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1735::DEBUG::2012-12-09 12:42:35,166::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1735::WARNING::2012-12-09 12:42:35,167::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1735::DEBUG::2012-12-09 12:42:35,168::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1735::WARNING::2012-12-09 12:42:35,168::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1735::INFO::2012-12-09 12:42:35,168::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1735::DEBUG::2012-12-09 12:42:35,169::task::1172::TaskManager.Task::(prepare) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1735::DEBUG::2012-12-09 12:42:35,169::task::588::TaskManager.Task::(_updateState) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::moving from state preparing -> state finished
>Thread-1735::DEBUG::2012-12-09 12:42:35,169::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1735::DEBUG::2012-12-09 12:42:35,169::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1735::DEBUG::2012-12-09 12:42:35,170::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1735::DEBUG::2012-12-09 12:42:35,170::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1735::DEBUG::2012-12-09 12:42:35,170::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1735::DEBUG::2012-12-09 12:42:35,170::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1735::DEBUG::2012-12-09 12:42:35,170::task::978::TaskManager.Task::(_decref) Task=`31315f92-bc79-4de6-9a75-c612cb401d03`::ref 0 aborting False
>Thread-1736::DEBUG::2012-12-09 12:42:35,475::task::588::TaskManager.Task::(_updateState) Task=`b1bb311e-e386-42da-9366-23c6cb2572a6`::moving from state init -> state preparing
>Thread-1736::INFO::2012-12-09 12:42:35,475::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1736::INFO::2012-12-09 12:42:35,475::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0079391002655', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1736::DEBUG::2012-12-09 12:42:35,476::task::1172::TaskManager.Task::(prepare) Task=`b1bb311e-e386-42da-9366-23c6cb2572a6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0079391002655', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-1736::DEBUG::2012-12-09 12:42:35,476::task::588::TaskManager.Task::(_updateState) Task=`b1bb311e-e386-42da-9366-23c6cb2572a6`::moving from state preparing -> state finished
>Thread-1736::DEBUG::2012-12-09 12:42:35,476::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1736::DEBUG::2012-12-09 12:42:35,476::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1736::DEBUG::2012-12-09 12:42:35,476::task::978::TaskManager.Task::(_decref) Task=`b1bb311e-e386-42da-9366-23c6cb2572a6`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:42:35,813::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:35,814::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:35,831::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:35,870::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165142 s, 62.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:42:35,911::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:35,912::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:35,912::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:35,913::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:37,877::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:37,916::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166105 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:39,922::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:39,950::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:39,959::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000253183 s, 16.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:39,984::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169005 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:41,993::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:42,032::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164891 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:44,039::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:44,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158163 s, 64.7 MB/s\n'; <rc> = 0
>Thread-1742::DEBUG::2012-12-09 12:42:45,283::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1742::DEBUG::2012-12-09 12:42:45,284::task::588::TaskManager.Task::(_updateState) Task=`f905892f-2e6a-44e8-94b3-6c1cebb9fd2b`::moving from state init -> state preparing
>Thread-1742::INFO::2012-12-09 12:42:45,284::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1742::INFO::2012-12-09 12:42:45,284::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1742::DEBUG::2012-12-09 12:42:45,285::task::1172::TaskManager.Task::(prepare) Task=`f905892f-2e6a-44e8-94b3-6c1cebb9fd2b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1742::DEBUG::2012-12-09 12:42:45,285::task::588::TaskManager.Task::(_updateState) Task=`f905892f-2e6a-44e8-94b3-6c1cebb9fd2b`::moving from state preparing -> state finished
>Thread-1742::DEBUG::2012-12-09 12:42:45,285::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1742::DEBUG::2012-12-09 12:42:45,285::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1742::DEBUG::2012-12-09 12:42:45,285::task::978::TaskManager.Task::(_decref) Task=`f905892f-2e6a-44e8-94b3-6c1cebb9fd2b`::ref 0 aborting False
>Thread-1743::DEBUG::2012-12-09 12:42:45,297::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1743::DEBUG::2012-12-09 12:42:45,298::task::588::TaskManager.Task::(_updateState) Task=`60523547-ad89-499a-9089-246d2cdc4469`::moving from state init -> state preparing
>Thread-1743::INFO::2012-12-09 12:42:45,298::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1743::DEBUG::2012-12-09 12:42:45,298::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a377114-9b7f-437e-ab93-82f7d2538324`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1743::DEBUG::2012-12-09 12:42:45,299::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1743::DEBUG::2012-12-09 12:42:45,299::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1743::DEBUG::2012-12-09 12:42:45,299::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a377114-9b7f-437e-ab93-82f7d2538324`::Granted request
>Thread-1743::DEBUG::2012-12-09 12:42:45,299::task::817::TaskManager.Task::(resourceAcquired) Task=`60523547-ad89-499a-9089-246d2cdc4469`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1743::DEBUG::2012-12-09 12:42:45,300::task::978::TaskManager.Task::(_decref) Task=`60523547-ad89-499a-9089-246d2cdc4469`::ref 1 aborting False
>Thread-1743::DEBUG::2012-12-09 12:42:45,300::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1743::DEBUG::2012-12-09 12:42:45,301::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1743::DEBUG::2012-12-09 12:42:45,397::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1743::WARNING::2012-12-09 12:42:45,398::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1743::DEBUG::2012-12-09 12:42:45,399::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1743::WARNING::2012-12-09 12:42:45,399::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1743::INFO::2012-12-09 12:42:45,399::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1743::DEBUG::2012-12-09 12:42:45,400::task::1172::TaskManager.Task::(prepare) Task=`60523547-ad89-499a-9089-246d2cdc4469`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1743::DEBUG::2012-12-09 12:42:45,400::task::588::TaskManager.Task::(_updateState) Task=`60523547-ad89-499a-9089-246d2cdc4469`::moving from state preparing -> state finished
>Thread-1743::DEBUG::2012-12-09 12:42:45,400::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1743::DEBUG::2012-12-09 12:42:45,400::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1743::DEBUG::2012-12-09 12:42:45,401::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1743::DEBUG::2012-12-09 12:42:45,401::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1743::DEBUG::2012-12-09 12:42:45,401::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1743::DEBUG::2012-12-09 12:42:45,401::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1743::DEBUG::2012-12-09 12:42:45,401::task::978::TaskManager.Task::(_decref) Task=`60523547-ad89-499a-9089-246d2cdc4469`::ref 0 aborting False
>Thread-1744::DEBUG::2012-12-09 12:42:45,680::task::588::TaskManager.Task::(_updateState) Task=`cdfd6954-cbe8-4fc8-a7b9-870b6467bde7`::moving from state init -> state preparing
>Thread-1744::INFO::2012-12-09 12:42:45,681::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1744::INFO::2012-12-09 12:42:45,681::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0106439590454', 'lastCheck': '5.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-1744::DEBUG::2012-12-09 12:42:45,681::task::1172::TaskManager.Task::(prepare) Task=`cdfd6954-cbe8-4fc8-a7b9-870b6467bde7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0106439590454', 'lastCheck': '5.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-1744::DEBUG::2012-12-09 12:42:45,682::task::588::TaskManager.Task::(_updateState) Task=`cdfd6954-cbe8-4fc8-a7b9-870b6467bde7`::moving from state preparing -> state finished
>Thread-1744::DEBUG::2012-12-09 12:42:45,682::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1744::DEBUG::2012-12-09 12:42:45,682::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1744::DEBUG::2012-12-09 12:42:45,682::task::978::TaskManager.Task::(_decref) Task=`cdfd6954-cbe8-4fc8-a7b9-870b6467bde7`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:42:45,914::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:45,915::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:42:46,012::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:46,013::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:46,013::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:46,013::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:46,084::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:46,123::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164334 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:48,131::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:48,174::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199332 s, 51.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:42:49,967::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:49,981::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000340997 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:50,181::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:50,222::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167211 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:52,228::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:52,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211914 s, 48.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:54,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:54,324::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205182 s, 49.9 MB/s\n'; <rc> = 0
>Thread-1750::DEBUG::2012-12-09 12:42:55,459::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1750::DEBUG::2012-12-09 12:42:55,460::task::588::TaskManager.Task::(_updateState) Task=`d398cbb9-def2-4f67-918d-926621326432`::moving from state init -> state preparing
>Thread-1750::INFO::2012-12-09 12:42:55,460::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1750::INFO::2012-12-09 12:42:55,461::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1750::DEBUG::2012-12-09 12:42:55,461::task::1172::TaskManager.Task::(prepare) Task=`d398cbb9-def2-4f67-918d-926621326432`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1750::DEBUG::2012-12-09 12:42:55,461::task::588::TaskManager.Task::(_updateState) Task=`d398cbb9-def2-4f67-918d-926621326432`::moving from state preparing -> state finished
>Thread-1750::DEBUG::2012-12-09 12:42:55,461::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1750::DEBUG::2012-12-09 12:42:55,461::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1750::DEBUG::2012-12-09 12:42:55,462::task::978::TaskManager.Task::(_decref) Task=`d398cbb9-def2-4f67-918d-926621326432`::ref 0 aborting False
>Thread-1751::DEBUG::2012-12-09 12:42:55,475::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1751::DEBUG::2012-12-09 12:42:55,476::task::588::TaskManager.Task::(_updateState) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::moving from state init -> state preparing
>Thread-1751::INFO::2012-12-09 12:42:55,476::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1751::DEBUG::2012-12-09 12:42:55,476::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2f8d3710-8e59-48ed-9c25-73ac18f542bd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1751::DEBUG::2012-12-09 12:42:55,476::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1751::DEBUG::2012-12-09 12:42:55,477::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1751::DEBUG::2012-12-09 12:42:55,477::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2f8d3710-8e59-48ed-9c25-73ac18f542bd`::Granted request
>Thread-1751::DEBUG::2012-12-09 12:42:55,477::task::817::TaskManager.Task::(resourceAcquired) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1751::DEBUG::2012-12-09 12:42:55,478::task::978::TaskManager.Task::(_decref) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::ref 1 aborting False
>Thread-1751::DEBUG::2012-12-09 12:42:55,479::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1751::DEBUG::2012-12-09 12:42:55,479::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1751::DEBUG::2012-12-09 12:42:55,580::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1751::WARNING::2012-12-09 12:42:55,581::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1751::DEBUG::2012-12-09 12:42:55,581::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1751::WARNING::2012-12-09 12:42:55,581::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1751::INFO::2012-12-09 12:42:55,582::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1751::DEBUG::2012-12-09 12:42:55,582::task::1172::TaskManager.Task::(prepare) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1751::DEBUG::2012-12-09 12:42:55,582::task::588::TaskManager.Task::(_updateState) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::moving from state preparing -> state finished
>Thread-1751::DEBUG::2012-12-09 12:42:55,583::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1751::DEBUG::2012-12-09 12:42:55,583::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1751::DEBUG::2012-12-09 12:42:55,583::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1751::DEBUG::2012-12-09 12:42:55,583::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1751::DEBUG::2012-12-09 12:42:55,583::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1751::DEBUG::2012-12-09 12:42:55,584::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1751::DEBUG::2012-12-09 12:42:55,584::task::978::TaskManager.Task::(_decref) Task=`0989c5fd-c5fa-4ab2-aef7-e4fe85c7a505`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:42:56,014::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:42:56,015::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1752::DEBUG::2012-12-09 12:42:56,052::task::588::TaskManager.Task::(_updateState) Task=`887d6d90-88c3-4735-8cc1-75e9f91f4328`::moving from state init -> state preparing
>Thread-1752::INFO::2012-12-09 12:42:56,053::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1752::INFO::2012-12-09 12:42:56,053::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.01407289505', 'lastCheck': '6.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-1752::DEBUG::2012-12-09 12:42:56,053::task::1172::TaskManager.Task::(prepare) Task=`887d6d90-88c3-4735-8cc1-75e9f91f4328`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.01407289505', 'lastCheck': '6.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-1752::DEBUG::2012-12-09 12:42:56,053::task::588::TaskManager.Task::(_updateState) Task=`887d6d90-88c3-4735-8cc1-75e9f91f4328`::moving from state preparing -> state finished
>Thread-1752::DEBUG::2012-12-09 12:42:56,054::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1752::DEBUG::2012-12-09 12:42:56,054::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1752::DEBUG::2012-12-09 12:42:56,054::task::978::TaskManager.Task::(_decref) Task=`887d6d90-88c3-4735-8cc1-75e9f91f4328`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:42:56,118::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:42:56,119::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:42:56,119::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:42:56,119::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:42:56,331::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:56,370::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163071 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:42:58,377::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:42:58,415::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158453 s, 64.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:42:59,984::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:42:59,995::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000497425 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:00,422::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:00,460::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156432 s, 65.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:02,467::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:02,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166657 s, 61.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:04,513::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:04,554::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165064 s, 62.0 MB/s\n'; <rc> = 0
>Thread-1758::DEBUG::2012-12-09 12:43:05,929::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1758::DEBUG::2012-12-09 12:43:05,930::task::588::TaskManager.Task::(_updateState) Task=`516d3324-3f97-4594-9ffd-c82999b7c32b`::moving from state init -> state preparing
>Thread-1758::INFO::2012-12-09 12:43:05,930::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1758::INFO::2012-12-09 12:43:05,930::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1758::DEBUG::2012-12-09 12:43:05,930::task::1172::TaskManager.Task::(prepare) Task=`516d3324-3f97-4594-9ffd-c82999b7c32b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1758::DEBUG::2012-12-09 12:43:05,931::task::588::TaskManager.Task::(_updateState) Task=`516d3324-3f97-4594-9ffd-c82999b7c32b`::moving from state preparing -> state finished
>Thread-1758::DEBUG::2012-12-09 12:43:05,931::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1758::DEBUG::2012-12-09 12:43:05,931::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1758::DEBUG::2012-12-09 12:43:05,931::task::978::TaskManager.Task::(_decref) Task=`516d3324-3f97-4594-9ffd-c82999b7c32b`::ref 0 aborting False
>Thread-1759::DEBUG::2012-12-09 12:43:05,943::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1759::DEBUG::2012-12-09 12:43:05,944::task::588::TaskManager.Task::(_updateState) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::moving from state init -> state preparing
>Thread-1759::INFO::2012-12-09 12:43:05,944::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1759::DEBUG::2012-12-09 12:43:05,945::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`77790a82-8ddf-4790-b1d2-c61da9f92339`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1759::DEBUG::2012-12-09 12:43:05,945::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1759::DEBUG::2012-12-09 12:43:05,945::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1759::DEBUG::2012-12-09 12:43:05,945::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`77790a82-8ddf-4790-b1d2-c61da9f92339`::Granted request
>Thread-1759::DEBUG::2012-12-09 12:43:05,945::task::817::TaskManager.Task::(resourceAcquired) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1759::DEBUG::2012-12-09 12:43:05,946::task::978::TaskManager.Task::(_decref) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::ref 1 aborting False
>Thread-1759::DEBUG::2012-12-09 12:43:05,946::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1759::DEBUG::2012-12-09 12:43:05,947::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1759::DEBUG::2012-12-09 12:43:06,042::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1759::WARNING::2012-12-09 12:43:06,043::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1759::DEBUG::2012-12-09 12:43:06,043::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1759::WARNING::2012-12-09 12:43:06,043::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1759::INFO::2012-12-09 12:43:06,044::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1759::DEBUG::2012-12-09 12:43:06,044::task::1172::TaskManager.Task::(prepare) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1759::DEBUG::2012-12-09 12:43:06,044::task::588::TaskManager.Task::(_updateState) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::moving from state preparing -> state finished
>Thread-1759::DEBUG::2012-12-09 12:43:06,044::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1759::DEBUG::2012-12-09 12:43:06,045::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1759::DEBUG::2012-12-09 12:43:06,045::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1759::DEBUG::2012-12-09 12:43:06,045::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1759::DEBUG::2012-12-09 12:43:06,045::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1759::DEBUG::2012-12-09 12:43:06,046::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1759::DEBUG::2012-12-09 12:43:06,046::task::978::TaskManager.Task::(_decref) Task=`6a2d2338-9760-4b35-9c5e-e885a38022ce`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:43:06,120::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:06,121::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:43:06,225::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:06,226::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:06,226::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:06,227::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1760::DEBUG::2012-12-09 12:43:06,440::task::588::TaskManager.Task::(_updateState) Task=`8d42bbb4-3876-453b-9647-04d0c90447b6`::moving from state init -> state preparing
>Thread-1760::INFO::2012-12-09 12:43:06,441::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1760::INFO::2012-12-09 12:43:06,441::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116150379181', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1760::DEBUG::2012-12-09 12:43:06,441::task::1172::TaskManager.Task::(prepare) Task=`8d42bbb4-3876-453b-9647-04d0c90447b6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116150379181', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-1760::DEBUG::2012-12-09 12:43:06,441::task::588::TaskManager.Task::(_updateState) Task=`8d42bbb4-3876-453b-9647-04d0c90447b6`::moving from state preparing -> state finished
>Thread-1760::DEBUG::2012-12-09 12:43:06,441::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1760::DEBUG::2012-12-09 12:43:06,442::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1760::DEBUG::2012-12-09 12:43:06,442::task::978::TaskManager.Task::(_decref) Task=`8d42bbb4-3876-453b-9647-04d0c90447b6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:06,561::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:06,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193668 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:08,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:08,649::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016482 s, 62.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:43:09,998::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:43:10,010::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000864102 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:10,656::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:10,700::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167613 s, 61.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:12,707::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:12,754::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219205 s, 46.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:14,761::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:14,811::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257223 s, 39.8 MB/s\n'; <rc> = 0
>Thread-1766::DEBUG::2012-12-09 12:43:16,189::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1766::DEBUG::2012-12-09 12:43:16,190::task::588::TaskManager.Task::(_updateState) Task=`82b3d92c-7a13-4947-a06a-d407b37385d1`::moving from state init -> state preparing
>Thread-1766::INFO::2012-12-09 12:43:16,191::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1766::INFO::2012-12-09 12:43:16,191::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1766::DEBUG::2012-12-09 12:43:16,192::task::1172::TaskManager.Task::(prepare) Task=`82b3d92c-7a13-4947-a06a-d407b37385d1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1766::DEBUG::2012-12-09 12:43:16,192::task::588::TaskManager.Task::(_updateState) Task=`82b3d92c-7a13-4947-a06a-d407b37385d1`::moving from state preparing -> state finished
>Thread-1766::DEBUG::2012-12-09 12:43:16,192::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1766::DEBUG::2012-12-09 12:43:16,193::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1766::DEBUG::2012-12-09 12:43:16,193::task::978::TaskManager.Task::(_decref) Task=`82b3d92c-7a13-4947-a06a-d407b37385d1`::ref 0 aborting False
>Thread-1767::DEBUG::2012-12-09 12:43:16,207::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1767::DEBUG::2012-12-09 12:43:16,208::task::588::TaskManager.Task::(_updateState) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::moving from state init -> state preparing
>Thread-1767::INFO::2012-12-09 12:43:16,208::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1767::DEBUG::2012-12-09 12:43:16,209::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ce7e6040-fd1c-40f4-8923-ce3a10178a3d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1767::DEBUG::2012-12-09 12:43:16,209::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1767::DEBUG::2012-12-09 12:43:16,209::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1767::DEBUG::2012-12-09 12:43:16,210::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ce7e6040-fd1c-40f4-8923-ce3a10178a3d`::Granted request
>Thread-1767::DEBUG::2012-12-09 12:43:16,210::task::817::TaskManager.Task::(resourceAcquired) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1767::DEBUG::2012-12-09 12:43:16,210::task::978::TaskManager.Task::(_decref) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::ref 1 aborting False
>Thread-1767::DEBUG::2012-12-09 12:43:16,211::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1767::DEBUG::2012-12-09 12:43:16,212::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1767::DEBUG::2012-12-09 12:43:16,321::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1767::WARNING::2012-12-09 12:43:16,323::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1767::DEBUG::2012-12-09 12:43:16,324::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:16,325::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:16,326::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1767::WARNING::2012-12-09 12:43:16,331::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1767::INFO::2012-12-09 12:43:16,332::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1767::DEBUG::2012-12-09 12:43:16,332::task::1172::TaskManager.Task::(prepare) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1767::DEBUG::2012-12-09 12:43:16,332::task::588::TaskManager.Task::(_updateState) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::moving from state preparing -> state finished
>Thread-1767::DEBUG::2012-12-09 12:43:16,332::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1767::DEBUG::2012-12-09 12:43:16,333::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1767::DEBUG::2012-12-09 12:43:16,333::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1767::DEBUG::2012-12-09 12:43:16,333::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1767::DEBUG::2012-12-09 12:43:16,333::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1767::DEBUG::2012-12-09 12:43:16,333::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1767::DEBUG::2012-12-09 12:43:16,334::task::978::TaskManager.Task::(_decref) Task=`9fce5518-bb6f-430d-8ff1-47d9523b6e5b`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:43:16,427::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:16,428::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:16,428::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:16,428::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:43:16,818::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:16,857::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165835 s, 61.7 MB/s\n'; <rc> = 0
>Thread-1768::DEBUG::2012-12-09 12:43:17,557::task::588::TaskManager.Task::(_updateState) Task=`66ea77bc-381a-431e-95c1-bda5498999dd`::moving from state init -> state preparing
>Thread-1768::INFO::2012-12-09 12:43:17,557::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1768::INFO::2012-12-09 12:43:17,557::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130240917206', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-1768::DEBUG::2012-12-09 12:43:17,558::task::1172::TaskManager.Task::(prepare) Task=`66ea77bc-381a-431e-95c1-bda5498999dd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130240917206', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-1768::DEBUG::2012-12-09 12:43:17,558::task::588::TaskManager.Task::(_updateState) Task=`66ea77bc-381a-431e-95c1-bda5498999dd`::moving from state preparing -> state finished
>Thread-1768::DEBUG::2012-12-09 12:43:17,558::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1768::DEBUG::2012-12-09 12:43:17,558::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1768::DEBUG::2012-12-09 12:43:17,558::task::978::TaskManager.Task::(_decref) Task=`66ea77bc-381a-431e-95c1-bda5498999dd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:18,863::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:18,902::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159427 s, 64.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:43:20,013::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:43:20,028::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0007954 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:20,909::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:20,948::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158927 s, 64.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:22,955::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:22,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200198 s, 51.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:25,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:25,047::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178509 s, 57.4 MB/s\n'; <rc> = 0
>Thread-1774::DEBUG::2012-12-09 12:43:26,422::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1774::DEBUG::2012-12-09 12:43:26,423::task::588::TaskManager.Task::(_updateState) Task=`b11974f5-7467-4c49-b360-c0f680ec8108`::moving from state init -> state preparing
>Thread-1774::INFO::2012-12-09 12:43:26,423::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1774::INFO::2012-12-09 12:43:26,423::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1774::DEBUG::2012-12-09 12:43:26,423::task::1172::TaskManager.Task::(prepare) Task=`b11974f5-7467-4c49-b360-c0f680ec8108`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1774::DEBUG::2012-12-09 12:43:26,424::task::588::TaskManager.Task::(_updateState) Task=`b11974f5-7467-4c49-b360-c0f680ec8108`::moving from state preparing -> state finished
>Thread-1774::DEBUG::2012-12-09 12:43:26,424::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1774::DEBUG::2012-12-09 12:43:26,424::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1774::DEBUG::2012-12-09 12:43:26,424::task::978::TaskManager.Task::(_decref) Task=`b11974f5-7467-4c49-b360-c0f680ec8108`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:43:26,429::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:26,430::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1775::DEBUG::2012-12-09 12:43:26,438::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1775::DEBUG::2012-12-09 12:43:26,439::task::588::TaskManager.Task::(_updateState) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::moving from state init -> state preparing
>Thread-1775::INFO::2012-12-09 12:43:26,439::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1775::DEBUG::2012-12-09 12:43:26,440::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`74b120eb-ce80-447d-8ad4-21d8e02a564e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1775::DEBUG::2012-12-09 12:43:26,440::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1775::DEBUG::2012-12-09 12:43:26,440::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1775::DEBUG::2012-12-09 12:43:26,440::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`74b120eb-ce80-447d-8ad4-21d8e02a564e`::Granted request
>Thread-1775::DEBUG::2012-12-09 12:43:26,441::task::817::TaskManager.Task::(resourceAcquired) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1775::DEBUG::2012-12-09 12:43:26,441::task::978::TaskManager.Task::(_decref) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 12:43:26,532::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:26,533::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:26,533::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:26,534::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1775::DEBUG::2012-12-09 12:43:26,535::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1775::DEBUG::2012-12-09 12:43:26,536::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1775::DEBUG::2012-12-09 12:43:26,635::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1775::WARNING::2012-12-09 12:43:26,637::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1775::DEBUG::2012-12-09 12:43:26,637::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1775::WARNING::2012-12-09 12:43:26,638::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1775::INFO::2012-12-09 12:43:26,638::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1775::DEBUG::2012-12-09 12:43:26,638::task::1172::TaskManager.Task::(prepare) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1775::DEBUG::2012-12-09 12:43:26,639::task::588::TaskManager.Task::(_updateState) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::moving from state preparing -> state finished
>Thread-1775::DEBUG::2012-12-09 12:43:26,639::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1775::DEBUG::2012-12-09 12:43:26,639::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1775::DEBUG::2012-12-09 12:43:26,639::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1775::DEBUG::2012-12-09 12:43:26,639::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1775::DEBUG::2012-12-09 12:43:26,640::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1775::DEBUG::2012-12-09 12:43:26,640::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1775::DEBUG::2012-12-09 12:43:26,640::task::978::TaskManager.Task::(_decref) Task=`a821167c-7708-45e3-9259-a464ccc0718c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:27,053::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:27,095::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167516 s, 61.1 MB/s\n'; <rc> = 0
>Thread-1776::DEBUG::2012-12-09 12:43:28,208::task::588::TaskManager.Task::(_updateState) Task=`b996734e-ed7f-4e14-b6d5-7ce9db47dfb5`::moving from state init -> state preparing
>Thread-1776::INFO::2012-12-09 12:43:28,208::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1776::INFO::2012-12-09 12:43:28,208::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156879425049', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-1776::DEBUG::2012-12-09 12:43:28,209::task::1172::TaskManager.Task::(prepare) Task=`b996734e-ed7f-4e14-b6d5-7ce9db47dfb5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156879425049', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-1776::DEBUG::2012-12-09 12:43:28,209::task::588::TaskManager.Task::(_updateState) Task=`b996734e-ed7f-4e14-b6d5-7ce9db47dfb5`::moving from state preparing -> state finished
>Thread-1776::DEBUG::2012-12-09 12:43:28,209::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1776::DEBUG::2012-12-09 12:43:28,209::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1776::DEBUG::2012-12-09 12:43:28,209::task::978::TaskManager.Task::(_decref) Task=`b996734e-ed7f-4e14-b6d5-7ce9db47dfb5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:29,102::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:29,152::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245946 s, 41.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:43:30,034::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:43:30,047::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000781414 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:31,159::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:31,205::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203224 s, 50.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:33,212::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:33,252::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168174 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:35,259::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:35,311::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266252 s, 38.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:43:36,536::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:36,537::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:43:36,646::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:36,647::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:36,647::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:36,648::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1782::DEBUG::2012-12-09 12:43:37,081::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1782::DEBUG::2012-12-09 12:43:37,082::task::588::TaskManager.Task::(_updateState) Task=`f556d73b-ed63-4b78-b32d-33fa31e7e83b`::moving from state init -> state preparing
>Thread-1782::INFO::2012-12-09 12:43:37,082::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1782::INFO::2012-12-09 12:43:37,083::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1782::DEBUG::2012-12-09 12:43:37,083::task::1172::TaskManager.Task::(prepare) Task=`f556d73b-ed63-4b78-b32d-33fa31e7e83b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1782::DEBUG::2012-12-09 12:43:37,083::task::588::TaskManager.Task::(_updateState) Task=`f556d73b-ed63-4b78-b32d-33fa31e7e83b`::moving from state preparing -> state finished
>Thread-1782::DEBUG::2012-12-09 12:43:37,084::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1782::DEBUG::2012-12-09 12:43:37,084::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1782::DEBUG::2012-12-09 12:43:37,084::task::978::TaskManager.Task::(_decref) Task=`f556d73b-ed63-4b78-b32d-33fa31e7e83b`::ref 0 aborting False
>Thread-1783::DEBUG::2012-12-09 12:43:37,101::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1783::DEBUG::2012-12-09 12:43:37,102::task::588::TaskManager.Task::(_updateState) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::moving from state init -> state preparing
>Thread-1783::INFO::2012-12-09 12:43:37,102::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1783::DEBUG::2012-12-09 12:43:37,103::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e9764d95-e66e-4229-b285-d89235e6ed4d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1783::DEBUG::2012-12-09 12:43:37,104::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1783::DEBUG::2012-12-09 12:43:37,104::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1783::DEBUG::2012-12-09 12:43:37,105::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e9764d95-e66e-4229-b285-d89235e6ed4d`::Granted request
>Thread-1783::DEBUG::2012-12-09 12:43:37,105::task::817::TaskManager.Task::(resourceAcquired) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1783::DEBUG::2012-12-09 12:43:37,106::task::978::TaskManager.Task::(_decref) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::ref 1 aborting False
>Thread-1783::DEBUG::2012-12-09 12:43:37,107::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1783::DEBUG::2012-12-09 12:43:37,107::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1783::DEBUG::2012-12-09 12:43:37,203::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1783::WARNING::2012-12-09 12:43:37,204::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1783::DEBUG::2012-12-09 12:43:37,204::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1783::WARNING::2012-12-09 12:43:37,205::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1783::INFO::2012-12-09 12:43:37,205::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1783::DEBUG::2012-12-09 12:43:37,206::task::1172::TaskManager.Task::(prepare) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1783::DEBUG::2012-12-09 12:43:37,206::task::588::TaskManager.Task::(_updateState) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::moving from state preparing -> state finished
>Thread-1783::DEBUG::2012-12-09 12:43:37,206::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1783::DEBUG::2012-12-09 12:43:37,206::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1783::DEBUG::2012-12-09 12:43:37,207::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1783::DEBUG::2012-12-09 12:43:37,208::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1783::DEBUG::2012-12-09 12:43:37,208::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1783::DEBUG::2012-12-09 12:43:37,208::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1783::DEBUG::2012-12-09 12:43:37,209::task::978::TaskManager.Task::(_decref) Task=`9d95a0a7-0c7b-4b81-9bfc-92666ec0cf1b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:37,318::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:37,359::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175696 s, 58.3 MB/s\n'; <rc> = 0
>Thread-1784::DEBUG::2012-12-09 12:43:38,603::task::588::TaskManager.Task::(_updateState) Task=`75a72c18-7f2e-4029-8360-ab1f74bc7273`::moving from state init -> state preparing
>Thread-1784::INFO::2012-12-09 12:43:38,603::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1784::INFO::2012-12-09 12:43:38,603::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138010978699', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-1784::DEBUG::2012-12-09 12:43:38,603::task::1172::TaskManager.Task::(prepare) Task=`75a72c18-7f2e-4029-8360-ab1f74bc7273`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138010978699', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-1784::DEBUG::2012-12-09 12:43:38,604::task::588::TaskManager.Task::(_updateState) Task=`75a72c18-7f2e-4029-8360-ab1f74bc7273`::moving from state preparing -> state finished
>Thread-1784::DEBUG::2012-12-09 12:43:38,604::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1784::DEBUG::2012-12-09 12:43:38,604::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1784::DEBUG::2012-12-09 12:43:38,604::task::978::TaskManager.Task::(_decref) Task=`75a72c18-7f2e-4029-8360-ab1f74bc7273`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:39,365::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:39,403::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155036 s, 66.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:43:40,050::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:43:40,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00047566 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:41,409::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:41,452::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180257 s, 56.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:43,458::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:43,503::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205653 s, 49.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:45,509::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:45,553::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185777 s, 55.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:43:46,649::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:46,650::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:43:46,760::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:46,761::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:46,762::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:46,762::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:43:47,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:47,606::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199183 s, 51.4 MB/s\n'; <rc> = 0
>Thread-1790::DEBUG::2012-12-09 12:43:47,651::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1790::DEBUG::2012-12-09 12:43:47,652::task::588::TaskManager.Task::(_updateState) Task=`b4dbfa3a-b5f1-4c66-829f-d340e665e037`::moving from state init -> state preparing
>Thread-1790::INFO::2012-12-09 12:43:47,652::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1790::INFO::2012-12-09 12:43:47,653::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1790::DEBUG::2012-12-09 12:43:47,654::task::1172::TaskManager.Task::(prepare) Task=`b4dbfa3a-b5f1-4c66-829f-d340e665e037`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1790::DEBUG::2012-12-09 12:43:47,654::task::588::TaskManager.Task::(_updateState) Task=`b4dbfa3a-b5f1-4c66-829f-d340e665e037`::moving from state preparing -> state finished
>Thread-1790::DEBUG::2012-12-09 12:43:47,655::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1790::DEBUG::2012-12-09 12:43:47,655::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1790::DEBUG::2012-12-09 12:43:47,656::task::978::TaskManager.Task::(_decref) Task=`b4dbfa3a-b5f1-4c66-829f-d340e665e037`::ref 0 aborting False
>Thread-1791::DEBUG::2012-12-09 12:43:47,672::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1791::DEBUG::2012-12-09 12:43:47,673::task::588::TaskManager.Task::(_updateState) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::moving from state init -> state preparing
>Thread-1791::INFO::2012-12-09 12:43:47,673::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1791::DEBUG::2012-12-09 12:43:47,674::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f5c8ad0-1067-4118-9c45-87691c0c7f80`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1791::DEBUG::2012-12-09 12:43:47,674::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1791::DEBUG::2012-12-09 12:43:47,674::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1791::DEBUG::2012-12-09 12:43:47,674::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5f5c8ad0-1067-4118-9c45-87691c0c7f80`::Granted request
>Thread-1791::DEBUG::2012-12-09 12:43:47,675::task::817::TaskManager.Task::(resourceAcquired) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1791::DEBUG::2012-12-09 12:43:47,675::task::978::TaskManager.Task::(_decref) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::ref 1 aborting False
>Thread-1791::DEBUG::2012-12-09 12:43:47,676::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1791::DEBUG::2012-12-09 12:43:47,676::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1791::DEBUG::2012-12-09 12:43:47,777::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1791::WARNING::2012-12-09 12:43:47,778::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1791::DEBUG::2012-12-09 12:43:47,778::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1791::WARNING::2012-12-09 12:43:47,778::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1791::INFO::2012-12-09 12:43:47,779::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1791::DEBUG::2012-12-09 12:43:47,779::task::1172::TaskManager.Task::(prepare) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1791::DEBUG::2012-12-09 12:43:47,779::task::588::TaskManager.Task::(_updateState) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::moving from state preparing -> state finished
>Thread-1791::DEBUG::2012-12-09 12:43:47,779::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1791::DEBUG::2012-12-09 12:43:47,780::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1791::DEBUG::2012-12-09 12:43:47,780::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1791::DEBUG::2012-12-09 12:43:47,780::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1791::DEBUG::2012-12-09 12:43:47,780::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1791::DEBUG::2012-12-09 12:43:47,780::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1791::DEBUG::2012-12-09 12:43:47,781::task::978::TaskManager.Task::(_decref) Task=`da719fc5-3c5b-4643-862c-494b4d564218`::ref 0 aborting False
>Thread-1792::DEBUG::2012-12-09 12:43:49,001::task::588::TaskManager.Task::(_updateState) Task=`692846de-7edc-4a40-aa07-5dd6605a5906`::moving from state init -> state preparing
>Thread-1792::INFO::2012-12-09 12:43:49,001::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1792::INFO::2012-12-09 12:43:49,001::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108368396759', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-1792::DEBUG::2012-12-09 12:43:49,001::task::1172::TaskManager.Task::(prepare) Task=`692846de-7edc-4a40-aa07-5dd6605a5906`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108368396759', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-1792::DEBUG::2012-12-09 12:43:49,002::task::588::TaskManager.Task::(_updateState) Task=`692846de-7edc-4a40-aa07-5dd6605a5906`::moving from state preparing -> state finished
>Thread-1792::DEBUG::2012-12-09 12:43:49,002::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1792::DEBUG::2012-12-09 12:43:49,002::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1792::DEBUG::2012-12-09 12:43:49,002::task::978::TaskManager.Task::(_decref) Task=`692846de-7edc-4a40-aa07-5dd6605a5906`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:49,613::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:49,652::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160449 s, 63.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:43:50,064::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:43:50,076::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000665172 s, 6.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:51,658::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:51,697::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168485 s, 60.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:53,704::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:53,757::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237867 s, 43.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:43:55,764::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:55,803::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164912 s, 62.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:43:56,763::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:43:56,764::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:43:56,860::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:43:56,862::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:43:56,862::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:43:56,862::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:43:57,810::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:57,859::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164352 s, 62.3 MB/s\n'; <rc> = 0
>Thread-1798::DEBUG::2012-12-09 12:43:58,562::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1798::DEBUG::2012-12-09 12:43:58,563::task::588::TaskManager.Task::(_updateState) Task=`667508b7-d9ad-4dbf-96fa-6e91b235d064`::moving from state init -> state preparing
>Thread-1798::INFO::2012-12-09 12:43:58,563::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1798::INFO::2012-12-09 12:43:58,563::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1798::DEBUG::2012-12-09 12:43:58,564::task::1172::TaskManager.Task::(prepare) Task=`667508b7-d9ad-4dbf-96fa-6e91b235d064`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1798::DEBUG::2012-12-09 12:43:58,564::task::588::TaskManager.Task::(_updateState) Task=`667508b7-d9ad-4dbf-96fa-6e91b235d064`::moving from state preparing -> state finished
>Thread-1798::DEBUG::2012-12-09 12:43:58,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1798::DEBUG::2012-12-09 12:43:58,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1798::DEBUG::2012-12-09 12:43:58,564::task::978::TaskManager.Task::(_decref) Task=`667508b7-d9ad-4dbf-96fa-6e91b235d064`::ref 0 aborting False
>Thread-1799::DEBUG::2012-12-09 12:43:58,577::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1799::DEBUG::2012-12-09 12:43:58,577::task::588::TaskManager.Task::(_updateState) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::moving from state init -> state preparing
>Thread-1799::INFO::2012-12-09 12:43:58,578::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1799::DEBUG::2012-12-09 12:43:58,578::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2334264e-abad-460e-8d49-7d354d1505ed`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1799::DEBUG::2012-12-09 12:43:58,578::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1799::DEBUG::2012-12-09 12:43:58,579::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1799::DEBUG::2012-12-09 12:43:58,579::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2334264e-abad-460e-8d49-7d354d1505ed`::Granted request
>Thread-1799::DEBUG::2012-12-09 12:43:58,579::task::817::TaskManager.Task::(resourceAcquired) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1799::DEBUG::2012-12-09 12:43:58,579::task::978::TaskManager.Task::(_decref) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::ref 1 aborting False
>Thread-1799::DEBUG::2012-12-09 12:43:58,580::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1799::DEBUG::2012-12-09 12:43:58,580::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1799::DEBUG::2012-12-09 12:43:58,677::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1799::WARNING::2012-12-09 12:43:58,678::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1799::DEBUG::2012-12-09 12:43:58,678::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1799::WARNING::2012-12-09 12:43:58,679::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1799::INFO::2012-12-09 12:43:58,679::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1799::DEBUG::2012-12-09 12:43:58,679::task::1172::TaskManager.Task::(prepare) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1799::DEBUG::2012-12-09 12:43:58,679::task::588::TaskManager.Task::(_updateState) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::moving from state preparing -> state finished
>Thread-1799::DEBUG::2012-12-09 12:43:58,680::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1799::DEBUG::2012-12-09 12:43:58,680::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1799::DEBUG::2012-12-09 12:43:58,680::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1799::DEBUG::2012-12-09 12:43:58,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1799::DEBUG::2012-12-09 12:43:58,681::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1799::DEBUG::2012-12-09 12:43:58,681::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1799::DEBUG::2012-12-09 12:43:58,681::task::978::TaskManager.Task::(_decref) Task=`86a63df5-88c1-4cb4-b4af-eae234183537`::ref 0 aborting False
>Thread-1800::DEBUG::2012-12-09 12:43:59,382::task::588::TaskManager.Task::(_updateState) Task=`96d91a04-02e1-427e-80ec-31f7c45d7c4a`::moving from state init -> state preparing
>Thread-1800::INFO::2012-12-09 12:43:59,383::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1800::INFO::2012-12-09 12:43:59,383::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129849910736', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-1800::DEBUG::2012-12-09 12:43:59,383::task::1172::TaskManager.Task::(prepare) Task=`96d91a04-02e1-427e-80ec-31f7c45d7c4a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129849910736', 'lastCheck': '9.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-1800::DEBUG::2012-12-09 12:43:59,383::task::588::TaskManager.Task::(_updateState) Task=`96d91a04-02e1-427e-80ec-31f7c45d7c4a`::moving from state preparing -> state finished
>Thread-1800::DEBUG::2012-12-09 12:43:59,384::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1800::DEBUG::2012-12-09 12:43:59,384::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1800::DEBUG::2012-12-09 12:43:59,384::task::978::TaskManager.Task::(_decref) Task=`96d91a04-02e1-427e-80ec-31f7c45d7c4a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:43:59,870::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:43:59,908::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157012 s, 65.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:00,079::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:00,089::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000507208 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:01,915::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:01,961::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235705 s, 43.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:03,967::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:04,007::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166336 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:06,014::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:06,062::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238094 s, 43.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:06,863::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:06,865::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:06,978::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:06,979::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:06,979::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:06,980::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:08,069::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:08,111::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017023 s, 60.2 MB/s\n'; <rc> = 0
>Thread-1806::DEBUG::2012-12-09 12:44:09,284::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1806::DEBUG::2012-12-09 12:44:09,284::task::588::TaskManager.Task::(_updateState) Task=`a610618a-3de9-4442-9ab0-09dc3370a7d5`::moving from state init -> state preparing
>Thread-1806::INFO::2012-12-09 12:44:09,285::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1806::INFO::2012-12-09 12:44:09,285::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1806::DEBUG::2012-12-09 12:44:09,286::task::1172::TaskManager.Task::(prepare) Task=`a610618a-3de9-4442-9ab0-09dc3370a7d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1806::DEBUG::2012-12-09 12:44:09,286::task::588::TaskManager.Task::(_updateState) Task=`a610618a-3de9-4442-9ab0-09dc3370a7d5`::moving from state preparing -> state finished
>Thread-1806::DEBUG::2012-12-09 12:44:09,286::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1806::DEBUG::2012-12-09 12:44:09,287::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1806::DEBUG::2012-12-09 12:44:09,287::task::978::TaskManager.Task::(_decref) Task=`a610618a-3de9-4442-9ab0-09dc3370a7d5`::ref 0 aborting False
>Thread-1807::DEBUG::2012-12-09 12:44:09,308::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1807::DEBUG::2012-12-09 12:44:09,308::task::588::TaskManager.Task::(_updateState) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::moving from state init -> state preparing
>Thread-1807::INFO::2012-12-09 12:44:09,309::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1807::DEBUG::2012-12-09 12:44:09,310::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9a27aa93-e5c4-404b-a036-d7107c7412d6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1807::DEBUG::2012-12-09 12:44:09,310::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1807::DEBUG::2012-12-09 12:44:09,311::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1807::DEBUG::2012-12-09 12:44:09,311::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9a27aa93-e5c4-404b-a036-d7107c7412d6`::Granted request
>Thread-1807::DEBUG::2012-12-09 12:44:09,312::task::817::TaskManager.Task::(resourceAcquired) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1807::DEBUG::2012-12-09 12:44:09,312::task::978::TaskManager.Task::(_decref) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::ref 1 aborting False
>Thread-1807::DEBUG::2012-12-09 12:44:09,313::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1807::DEBUG::2012-12-09 12:44:09,314::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1807::DEBUG::2012-12-09 12:44:09,430::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1807::WARNING::2012-12-09 12:44:09,431::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1807::DEBUG::2012-12-09 12:44:09,431::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1807::WARNING::2012-12-09 12:44:09,432::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1807::INFO::2012-12-09 12:44:09,432::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1807::DEBUG::2012-12-09 12:44:09,432::task::1172::TaskManager.Task::(prepare) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1807::DEBUG::2012-12-09 12:44:09,432::task::588::TaskManager.Task::(_updateState) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::moving from state preparing -> state finished
>Thread-1807::DEBUG::2012-12-09 12:44:09,433::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1807::DEBUG::2012-12-09 12:44:09,433::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1807::DEBUG::2012-12-09 12:44:09,433::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1807::DEBUG::2012-12-09 12:44:09,433::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1807::DEBUG::2012-12-09 12:44:09,433::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1807::DEBUG::2012-12-09 12:44:09,434::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1807::DEBUG::2012-12-09 12:44:09,434::task::978::TaskManager.Task::(_decref) Task=`32e4a32f-31ce-4ae1-818c-1dcf0e38ef99`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:44:10,092::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:44:10,092::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:44:10,092::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:10,099::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000465989 s, 4.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:10,100::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000465989 s, 4.4 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:44:10,100::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:44:10,100::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:44:10,100::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:44:10,100::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,101::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,101::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,101::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,101::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,102::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:10,117::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:10,156::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160738 s, 63.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:10,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:10,202::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:44:10,202::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:44:10,203::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:44:10,203::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:44:10,203::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:44:10,203::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:10,311::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:10,312::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:10,319::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000610806 s, 6.7 MB/s\n'; <rc> = 0
>Thread-1808::DEBUG::2012-12-09 12:44:10,539::task::588::TaskManager.Task::(_updateState) Task=`a5fd6fc4-3281-44f1-8970-5863d5efe790`::moving from state init -> state preparing
>Thread-1808::INFO::2012-12-09 12:44:10,540::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1808::INFO::2012-12-09 12:44:10,540::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00784587860107', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1808::DEBUG::2012-12-09 12:44:10,541::task::1172::TaskManager.Task::(prepare) Task=`a5fd6fc4-3281-44f1-8970-5863d5efe790`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00784587860107', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1808::DEBUG::2012-12-09 12:44:10,541::task::588::TaskManager.Task::(_updateState) Task=`a5fd6fc4-3281-44f1-8970-5863d5efe790`::moving from state preparing -> state finished
>Thread-1808::DEBUG::2012-12-09 12:44:10,542::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1808::DEBUG::2012-12-09 12:44:10,542::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1808::DEBUG::2012-12-09 12:44:10,542::task::978::TaskManager.Task::(_decref) Task=`a5fd6fc4-3281-44f1-8970-5863d5efe790`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:12,162::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:12,202::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170498 s, 60.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:14,208::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:14,255::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172991 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:16,265::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:16,303::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165968 s, 61.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:16,981::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:16,981::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:17,091::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:17,093::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:17,094::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:17,095::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:18,310::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:18,358::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225382 s, 45.4 MB/s\n'; <rc> = 0
>Thread-1814::DEBUG::2012-12-09 12:44:19,545::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1814::DEBUG::2012-12-09 12:44:19,545::task::588::TaskManager.Task::(_updateState) Task=`d9b4977d-1b4d-4d61-858a-8c1a62ddd8e4`::moving from state init -> state preparing
>Thread-1814::INFO::2012-12-09 12:44:19,546::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1814::INFO::2012-12-09 12:44:19,546::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1814::DEBUG::2012-12-09 12:44:19,546::task::1172::TaskManager.Task::(prepare) Task=`d9b4977d-1b4d-4d61-858a-8c1a62ddd8e4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1814::DEBUG::2012-12-09 12:44:19,546::task::588::TaskManager.Task::(_updateState) Task=`d9b4977d-1b4d-4d61-858a-8c1a62ddd8e4`::moving from state preparing -> state finished
>Thread-1814::DEBUG::2012-12-09 12:44:19,546::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1814::DEBUG::2012-12-09 12:44:19,547::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1814::DEBUG::2012-12-09 12:44:19,547::task::978::TaskManager.Task::(_decref) Task=`d9b4977d-1b4d-4d61-858a-8c1a62ddd8e4`::ref 0 aborting False
>Thread-1815::DEBUG::2012-12-09 12:44:19,563::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1815::DEBUG::2012-12-09 12:44:19,563::task::588::TaskManager.Task::(_updateState) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::moving from state init -> state preparing
>Thread-1815::INFO::2012-12-09 12:44:19,564::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1815::DEBUG::2012-12-09 12:44:19,564::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c620c951-a3cf-49b4-ba7f-61352c165b6e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1815::DEBUG::2012-12-09 12:44:19,564::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1815::DEBUG::2012-12-09 12:44:19,565::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1815::DEBUG::2012-12-09 12:44:19,565::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c620c951-a3cf-49b4-ba7f-61352c165b6e`::Granted request
>Thread-1815::DEBUG::2012-12-09 12:44:19,565::task::817::TaskManager.Task::(resourceAcquired) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1815::DEBUG::2012-12-09 12:44:19,565::task::978::TaskManager.Task::(_decref) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::ref 1 aborting False
>Thread-1815::DEBUG::2012-12-09 12:44:19,566::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1815::DEBUG::2012-12-09 12:44:19,566::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1815::DEBUG::2012-12-09 12:44:19,663::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1815::WARNING::2012-12-09 12:44:19,664::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1815::DEBUG::2012-12-09 12:44:19,664::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1815::WARNING::2012-12-09 12:44:19,664::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1815::INFO::2012-12-09 12:44:19,665::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1815::DEBUG::2012-12-09 12:44:19,665::task::1172::TaskManager.Task::(prepare) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1815::DEBUG::2012-12-09 12:44:19,665::task::588::TaskManager.Task::(_updateState) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::moving from state preparing -> state finished
>Thread-1815::DEBUG::2012-12-09 12:44:19,665::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1815::DEBUG::2012-12-09 12:44:19,666::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1815::DEBUG::2012-12-09 12:44:19,666::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1815::DEBUG::2012-12-09 12:44:19,666::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1815::DEBUG::2012-12-09 12:44:19,666::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1815::DEBUG::2012-12-09 12:44:19,667::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1815::DEBUG::2012-12-09 12:44:19,667::task::978::TaskManager.Task::(_decref) Task=`12ab2217-7bfb-4c47-bba8-6b5887137834`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:44:20,322::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:20,333::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000612822 s, 6.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:20,365::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:20,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217642 s, 47.0 MB/s\n'; <rc> = 0
>Thread-1816::DEBUG::2012-12-09 12:44:20,719::task::588::TaskManager.Task::(_updateState) Task=`5fef14b6-51b4-4246-b00d-57a35a1f56d7`::moving from state init -> state preparing
>Thread-1816::INFO::2012-12-09 12:44:20,720::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1816::INFO::2012-12-09 12:44:20,720::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112948417664', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1816::DEBUG::2012-12-09 12:44:20,720::task::1172::TaskManager.Task::(prepare) Task=`5fef14b6-51b4-4246-b00d-57a35a1f56d7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112948417664', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-1816::DEBUG::2012-12-09 12:44:20,721::task::588::TaskManager.Task::(_updateState) Task=`5fef14b6-51b4-4246-b00d-57a35a1f56d7`::moving from state preparing -> state finished
>Thread-1816::DEBUG::2012-12-09 12:44:20,721::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1816::DEBUG::2012-12-09 12:44:20,721::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1816::DEBUG::2012-12-09 12:44:20,721::task::978::TaskManager.Task::(_decref) Task=`5fef14b6-51b4-4246-b00d-57a35a1f56d7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:22,428::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:22,475::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220845 s, 46.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:24,482::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:24,526::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183447 s, 55.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:26,532::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:26,573::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162162 s, 63.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:27,096::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:44:27,097::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:27,097::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:27,197::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:27,198::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:27,198::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:27,198::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:28,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:28,619::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164785 s, 62.1 MB/s\n'; <rc> = 0
>Thread-1822::DEBUG::2012-12-09 12:44:29,961::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1822::DEBUG::2012-12-09 12:44:29,961::task::588::TaskManager.Task::(_updateState) Task=`bc952505-c10f-4a2a-bc05-297f405f7ba0`::moving from state init -> state preparing
>Thread-1822::INFO::2012-12-09 12:44:29,962::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1822::INFO::2012-12-09 12:44:29,962::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1822::DEBUG::2012-12-09 12:44:29,962::task::1172::TaskManager.Task::(prepare) Task=`bc952505-c10f-4a2a-bc05-297f405f7ba0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1822::DEBUG::2012-12-09 12:44:29,963::task::588::TaskManager.Task::(_updateState) Task=`bc952505-c10f-4a2a-bc05-297f405f7ba0`::moving from state preparing -> state finished
>Thread-1822::DEBUG::2012-12-09 12:44:29,963::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1822::DEBUG::2012-12-09 12:44:29,963::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1822::DEBUG::2012-12-09 12:44:29,964::task::978::TaskManager.Task::(_decref) Task=`bc952505-c10f-4a2a-bc05-297f405f7ba0`::ref 0 aborting False
>Thread-1823::DEBUG::2012-12-09 12:44:29,988::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1823::DEBUG::2012-12-09 12:44:29,989::task::588::TaskManager.Task::(_updateState) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::moving from state init -> state preparing
>Thread-1823::INFO::2012-12-09 12:44:29,989::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1823::DEBUG::2012-12-09 12:44:29,989::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2d1d5134-751c-4b5b-a280-f3737067bdfb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1823::DEBUG::2012-12-09 12:44:29,989::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1823::DEBUG::2012-12-09 12:44:29,990::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1823::DEBUG::2012-12-09 12:44:29,990::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2d1d5134-751c-4b5b-a280-f3737067bdfb`::Granted request
>Thread-1823::DEBUG::2012-12-09 12:44:29,990::task::817::TaskManager.Task::(resourceAcquired) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1823::DEBUG::2012-12-09 12:44:29,990::task::978::TaskManager.Task::(_decref) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::ref 1 aborting False
>Thread-1823::DEBUG::2012-12-09 12:44:29,991::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1823::DEBUG::2012-12-09 12:44:29,992::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1823::DEBUG::2012-12-09 12:44:30,093::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1823::WARNING::2012-12-09 12:44:30,094::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1823::DEBUG::2012-12-09 12:44:30,094::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1823::WARNING::2012-12-09 12:44:30,095::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1823::INFO::2012-12-09 12:44:30,095::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1823::DEBUG::2012-12-09 12:44:30,095::task::1172::TaskManager.Task::(prepare) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1823::DEBUG::2012-12-09 12:44:30,096::task::588::TaskManager.Task::(_updateState) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::moving from state preparing -> state finished
>Thread-1823::DEBUG::2012-12-09 12:44:30,096::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1823::DEBUG::2012-12-09 12:44:30,096::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1823::DEBUG::2012-12-09 12:44:30,097::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1823::DEBUG::2012-12-09 12:44:30,097::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1823::DEBUG::2012-12-09 12:44:30,097::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1823::DEBUG::2012-12-09 12:44:30,097::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1823::DEBUG::2012-12-09 12:44:30,097::task::978::TaskManager.Task::(_decref) Task=`c9385aec-1037-4790-a2ec-20c23cd73609`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:44:30,336::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:30,347::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000419765 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:30,626::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:30,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220082 s, 46.5 MB/s\n'; <rc> = 0
>Thread-1824::DEBUG::2012-12-09 12:44:31,010::task::588::TaskManager.Task::(_updateState) Task=`40c447eb-d3cf-4982-981b-f7cf420717a0`::moving from state init -> state preparing
>Thread-1824::INFO::2012-12-09 12:44:31,011::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1824::INFO::2012-12-09 12:44:31,011::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119521617889', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1824::DEBUG::2012-12-09 12:44:31,012::task::1172::TaskManager.Task::(prepare) Task=`40c447eb-d3cf-4982-981b-f7cf420717a0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119521617889', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-1824::DEBUG::2012-12-09 12:44:31,012::task::588::TaskManager.Task::(_updateState) Task=`40c447eb-d3cf-4982-981b-f7cf420717a0`::moving from state preparing -> state finished
>Thread-1824::DEBUG::2012-12-09 12:44:31,012::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1824::DEBUG::2012-12-09 12:44:31,013::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1824::DEBUG::2012-12-09 12:44:31,013::task::978::TaskManager.Task::(_decref) Task=`40c447eb-d3cf-4982-981b-f7cf420717a0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:32,681::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:32,725::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019347 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:34,732::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:34,776::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212159 s, 48.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:36,783::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:36,824::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170744 s, 60.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:37,200::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:37,201::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:37,305::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:37,306::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:37,306::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:37,307::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:38,830::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:38,880::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256393 s, 39.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:40,351::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:40,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000760867 s, 5.4 MB/s\n'; <rc> = 0
>Thread-1830::DEBUG::2012-12-09 12:44:40,432::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1830::DEBUG::2012-12-09 12:44:40,433::task::588::TaskManager.Task::(_updateState) Task=`bfecd18a-9689-497d-9148-4b4ac3e914e5`::moving from state init -> state preparing
>Thread-1830::INFO::2012-12-09 12:44:40,433::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1830::INFO::2012-12-09 12:44:40,434::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1830::DEBUG::2012-12-09 12:44:40,434::task::1172::TaskManager.Task::(prepare) Task=`bfecd18a-9689-497d-9148-4b4ac3e914e5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1830::DEBUG::2012-12-09 12:44:40,435::task::588::TaskManager.Task::(_updateState) Task=`bfecd18a-9689-497d-9148-4b4ac3e914e5`::moving from state preparing -> state finished
>Thread-1830::DEBUG::2012-12-09 12:44:40,435::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1830::DEBUG::2012-12-09 12:44:40,436::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1830::DEBUG::2012-12-09 12:44:40,436::task::978::TaskManager.Task::(_decref) Task=`bfecd18a-9689-497d-9148-4b4ac3e914e5`::ref 0 aborting False
>Thread-1831::DEBUG::2012-12-09 12:44:40,455::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1831::DEBUG::2012-12-09 12:44:40,455::task::588::TaskManager.Task::(_updateState) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::moving from state init -> state preparing
>Thread-1831::INFO::2012-12-09 12:44:40,455::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1831::DEBUG::2012-12-09 12:44:40,456::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0ad79f97-949c-4594-9760-bbcd459d3af3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1831::DEBUG::2012-12-09 12:44:40,456::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1831::DEBUG::2012-12-09 12:44:40,456::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1831::DEBUG::2012-12-09 12:44:40,457::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0ad79f97-949c-4594-9760-bbcd459d3af3`::Granted request
>Thread-1831::DEBUG::2012-12-09 12:44:40,457::task::817::TaskManager.Task::(resourceAcquired) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1831::DEBUG::2012-12-09 12:44:40,458::task::978::TaskManager.Task::(_decref) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::ref 1 aborting False
>Thread-1831::DEBUG::2012-12-09 12:44:40,459::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1831::DEBUG::2012-12-09 12:44:40,460::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1831::DEBUG::2012-12-09 12:44:40,578::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1831::WARNING::2012-12-09 12:44:40,581::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1831::DEBUG::2012-12-09 12:44:40,581::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1831::WARNING::2012-12-09 12:44:40,582::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1831::INFO::2012-12-09 12:44:40,583::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1831::DEBUG::2012-12-09 12:44:40,584::task::1172::TaskManager.Task::(prepare) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1831::DEBUG::2012-12-09 12:44:40,584::task::588::TaskManager.Task::(_updateState) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::moving from state preparing -> state finished
>Thread-1831::DEBUG::2012-12-09 12:44:40,584::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1831::DEBUG::2012-12-09 12:44:40,585::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1831::DEBUG::2012-12-09 12:44:40,585::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1831::DEBUG::2012-12-09 12:44:40,586::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1831::DEBUG::2012-12-09 12:44:40,586::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1831::DEBUG::2012-12-09 12:44:40,587::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1831::DEBUG::2012-12-09 12:44:40,587::task::978::TaskManager.Task::(_decref) Task=`d1834f33-4a3b-48d8-a242-2a7ee71a9d26`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:40,887::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:40,939::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275429 s, 37.2 MB/s\n'; <rc> = 0
>Thread-1832::DEBUG::2012-12-09 12:44:41,371::task::588::TaskManager.Task::(_updateState) Task=`0c9beac1-7aa4-4552-84db-c474605e5dd5`::moving from state init -> state preparing
>Thread-1832::INFO::2012-12-09 12:44:41,371::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1832::INFO::2012-12-09 12:44:41,372::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161571502686', 'lastCheck': '1.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1832::DEBUG::2012-12-09 12:44:41,372::task::1172::TaskManager.Task::(prepare) Task=`0c9beac1-7aa4-4552-84db-c474605e5dd5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0161571502686', 'lastCheck': '1.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1832::DEBUG::2012-12-09 12:44:41,373::task::588::TaskManager.Task::(_updateState) Task=`0c9beac1-7aa4-4552-84db-c474605e5dd5`::moving from state preparing -> state finished
>Thread-1832::DEBUG::2012-12-09 12:44:41,373::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1832::DEBUG::2012-12-09 12:44:41,374::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1832::DEBUG::2012-12-09 12:44:41,374::task::978::TaskManager.Task::(_decref) Task=`0c9beac1-7aa4-4552-84db-c474605e5dd5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:42,946::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:42,993::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218581 s, 46.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:45,000::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:45,052::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273242 s, 37.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:47,059::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:47,103::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194596 s, 52.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:47,308::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:47,309::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:47,419::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:47,422::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:47,422::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:47,423::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:49,109::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:49,152::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01976 s, 51.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:44:50,369::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:44:50,382::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000472604 s, 8.7 MB/s\n'; <rc> = 0
>Thread-1838::DEBUG::2012-12-09 12:44:50,654::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1838::DEBUG::2012-12-09 12:44:50,655::task::588::TaskManager.Task::(_updateState) Task=`b57f7959-5813-4d48-ba33-0c9b62ab2f2a`::moving from state init -> state preparing
>Thread-1838::INFO::2012-12-09 12:44:50,656::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1838::INFO::2012-12-09 12:44:50,656::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1838::DEBUG::2012-12-09 12:44:50,657::task::1172::TaskManager.Task::(prepare) Task=`b57f7959-5813-4d48-ba33-0c9b62ab2f2a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1838::DEBUG::2012-12-09 12:44:50,657::task::588::TaskManager.Task::(_updateState) Task=`b57f7959-5813-4d48-ba33-0c9b62ab2f2a`::moving from state preparing -> state finished
>Thread-1838::DEBUG::2012-12-09 12:44:50,657::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1838::DEBUG::2012-12-09 12:44:50,657::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1838::DEBUG::2012-12-09 12:44:50,657::task::978::TaskManager.Task::(_decref) Task=`b57f7959-5813-4d48-ba33-0c9b62ab2f2a`::ref 0 aborting False
>Thread-1839::DEBUG::2012-12-09 12:44:50,675::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1839::DEBUG::2012-12-09 12:44:50,675::task::588::TaskManager.Task::(_updateState) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::moving from state init -> state preparing
>Thread-1839::INFO::2012-12-09 12:44:50,676::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1839::DEBUG::2012-12-09 12:44:50,676::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6fc5d073-a80e-4157-a3fa-02a5f2c955f1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1839::DEBUG::2012-12-09 12:44:50,677::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1839::DEBUG::2012-12-09 12:44:50,677::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1839::DEBUG::2012-12-09 12:44:50,677::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6fc5d073-a80e-4157-a3fa-02a5f2c955f1`::Granted request
>Thread-1839::DEBUG::2012-12-09 12:44:50,677::task::817::TaskManager.Task::(resourceAcquired) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1839::DEBUG::2012-12-09 12:44:50,678::task::978::TaskManager.Task::(_decref) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::ref 1 aborting False
>Thread-1839::DEBUG::2012-12-09 12:44:50,678::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1839::DEBUG::2012-12-09 12:44:50,679::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1839::DEBUG::2012-12-09 12:44:50,782::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1839::WARNING::2012-12-09 12:44:50,783::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1839::DEBUG::2012-12-09 12:44:50,783::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1839::WARNING::2012-12-09 12:44:50,784::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1839::INFO::2012-12-09 12:44:50,784::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1839::DEBUG::2012-12-09 12:44:50,784::task::1172::TaskManager.Task::(prepare) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1839::DEBUG::2012-12-09 12:44:50,784::task::588::TaskManager.Task::(_updateState) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::moving from state preparing -> state finished
>Thread-1839::DEBUG::2012-12-09 12:44:50,785::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1839::DEBUG::2012-12-09 12:44:50,785::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1839::DEBUG::2012-12-09 12:44:50,785::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1839::DEBUG::2012-12-09 12:44:50,785::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1839::DEBUG::2012-12-09 12:44:50,785::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1839::DEBUG::2012-12-09 12:44:50,786::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1839::DEBUG::2012-12-09 12:44:50,786::task::978::TaskManager.Task::(_decref) Task=`dff60723-176d-4d33-b69b-ce73e67a691f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:51,158::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:51,199::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160193 s, 63.9 MB/s\n'; <rc> = 0
>Thread-1840::DEBUG::2012-12-09 12:44:51,616::task::588::TaskManager.Task::(_updateState) Task=`841c2e33-7902-4825-9c6a-f1e26ce7253f`::moving from state init -> state preparing
>Thread-1840::INFO::2012-12-09 12:44:51,617::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1840::INFO::2012-12-09 12:44:51,617::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130171775818', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-1840::DEBUG::2012-12-09 12:44:51,617::task::1172::TaskManager.Task::(prepare) Task=`841c2e33-7902-4825-9c6a-f1e26ce7253f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130171775818', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-1840::DEBUG::2012-12-09 12:44:51,617::task::588::TaskManager.Task::(_updateState) Task=`841c2e33-7902-4825-9c6a-f1e26ce7253f`::moving from state preparing -> state finished
>Thread-1840::DEBUG::2012-12-09 12:44:51,617::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1840::DEBUG::2012-12-09 12:44:51,618::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1840::DEBUG::2012-12-09 12:44:51,618::task::978::TaskManager.Task::(_decref) Task=`841c2e33-7902-4825-9c6a-f1e26ce7253f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:44:53,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:53,260::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028238 s, 36.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:55,266::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:55,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228905 s, 44.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:44:57,322::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:57,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170046 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:44:57,425::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:44:57,425::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:44:57,523::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:44:57,525::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:44:57,525::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:44:57,525::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:44:59,373::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:44:59,413::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163186 s, 62.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:00,386::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:00,395::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000594564 s, 6.9 MB/s\n'; <rc> = 0
>Thread-1846::DEBUG::2012-12-09 12:45:00,902::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1846::DEBUG::2012-12-09 12:45:00,903::task::588::TaskManager.Task::(_updateState) Task=`74c88346-f857-4af8-9bfe-22f0ade9e44f`::moving from state init -> state preparing
>Thread-1846::INFO::2012-12-09 12:45:00,904::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1846::INFO::2012-12-09 12:45:00,904::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1846::DEBUG::2012-12-09 12:45:00,905::task::1172::TaskManager.Task::(prepare) Task=`74c88346-f857-4af8-9bfe-22f0ade9e44f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1846::DEBUG::2012-12-09 12:45:00,905::task::588::TaskManager.Task::(_updateState) Task=`74c88346-f857-4af8-9bfe-22f0ade9e44f`::moving from state preparing -> state finished
>Thread-1846::DEBUG::2012-12-09 12:45:00,905::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1846::DEBUG::2012-12-09 12:45:00,906::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1846::DEBUG::2012-12-09 12:45:00,906::task::978::TaskManager.Task::(_decref) Task=`74c88346-f857-4af8-9bfe-22f0ade9e44f`::ref 0 aborting False
>Thread-1847::DEBUG::2012-12-09 12:45:00,930::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1847::DEBUG::2012-12-09 12:45:00,930::task::588::TaskManager.Task::(_updateState) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::moving from state init -> state preparing
>Thread-1847::INFO::2012-12-09 12:45:00,931::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1847::DEBUG::2012-12-09 12:45:00,932::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f9d17da2-ac9a-4ba3-b8c7-a7a4da992856`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1847::DEBUG::2012-12-09 12:45:00,932::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1847::DEBUG::2012-12-09 12:45:00,933::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1847::DEBUG::2012-12-09 12:45:00,933::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f9d17da2-ac9a-4ba3-b8c7-a7a4da992856`::Granted request
>Thread-1847::DEBUG::2012-12-09 12:45:00,934::task::817::TaskManager.Task::(resourceAcquired) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1847::DEBUG::2012-12-09 12:45:00,934::task::978::TaskManager.Task::(_decref) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::ref 1 aborting False
>Thread-1847::DEBUG::2012-12-09 12:45:00,936::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1847::DEBUG::2012-12-09 12:45:00,937::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1847::DEBUG::2012-12-09 12:45:01,047::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1847::WARNING::2012-12-09 12:45:01,050::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1847::DEBUG::2012-12-09 12:45:01,050::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1847::WARNING::2012-12-09 12:45:01,051::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1847::INFO::2012-12-09 12:45:01,052::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1847::DEBUG::2012-12-09 12:45:01,052::task::1172::TaskManager.Task::(prepare) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1847::DEBUG::2012-12-09 12:45:01,053::task::588::TaskManager.Task::(_updateState) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::moving from state preparing -> state finished
>Thread-1847::DEBUG::2012-12-09 12:45:01,053::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1847::DEBUG::2012-12-09 12:45:01,054::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1847::DEBUG::2012-12-09 12:45:01,054::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1847::DEBUG::2012-12-09 12:45:01,055::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1847::DEBUG::2012-12-09 12:45:01,055::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1847::DEBUG::2012-12-09 12:45:01,056::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1847::DEBUG::2012-12-09 12:45:01,056::task::978::TaskManager.Task::(_decref) Task=`d80a5db3-d0f3-4771-a6be-5696c7157cde`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:01,420::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:01,471::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259592 s, 39.4 MB/s\n'; <rc> = 0
>Thread-1848::DEBUG::2012-12-09 12:45:01,874::task::588::TaskManager.Task::(_updateState) Task=`f67e2f92-51de-49a8-b010-4ddcba8b20c7`::moving from state init -> state preparing
>Thread-1848::INFO::2012-12-09 12:45:01,874::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1848::INFO::2012-12-09 12:45:01,875::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00922513008118', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-1848::DEBUG::2012-12-09 12:45:01,875::task::1172::TaskManager.Task::(prepare) Task=`f67e2f92-51de-49a8-b010-4ddcba8b20c7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00922513008118', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-1848::DEBUG::2012-12-09 12:45:01,876::task::588::TaskManager.Task::(_updateState) Task=`f67e2f92-51de-49a8-b010-4ddcba8b20c7`::moving from state preparing -> state finished
>Thread-1848::DEBUG::2012-12-09 12:45:01,876::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1848::DEBUG::2012-12-09 12:45:01,877::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1848::DEBUG::2012-12-09 12:45:01,877::task::978::TaskManager.Task::(_decref) Task=`f67e2f92-51de-49a8-b010-4ddcba8b20c7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:03,479::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:03,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271831 s, 37.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:05,537::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:05,580::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017023 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:07,526::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:07,527::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:07,586::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:07,627::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:07,630::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:07,631::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:07,633::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:07,633::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173496 s, 59.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:09,640::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:09,690::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267569 s, 38.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:10,398::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:10,412::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000738289 s, 5.5 MB/s\n'; <rc> = 0
>Thread-1854::DEBUG::2012-12-09 12:45:11,670::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1854::DEBUG::2012-12-09 12:45:11,671::task::588::TaskManager.Task::(_updateState) Task=`682aec62-55e5-4b7c-8a90-00811733add7`::moving from state init -> state preparing
>Thread-1854::INFO::2012-12-09 12:45:11,672::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1854::INFO::2012-12-09 12:45:11,672::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1854::DEBUG::2012-12-09 12:45:11,673::task::1172::TaskManager.Task::(prepare) Task=`682aec62-55e5-4b7c-8a90-00811733add7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1854::DEBUG::2012-12-09 12:45:11,673::task::588::TaskManager.Task::(_updateState) Task=`682aec62-55e5-4b7c-8a90-00811733add7`::moving from state preparing -> state finished
>Thread-1854::DEBUG::2012-12-09 12:45:11,673::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1854::DEBUG::2012-12-09 12:45:11,674::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1854::DEBUG::2012-12-09 12:45:11,674::task::978::TaskManager.Task::(_decref) Task=`682aec62-55e5-4b7c-8a90-00811733add7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:11,697::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1855::DEBUG::2012-12-09 12:45:11,698::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1855::DEBUG::2012-12-09 12:45:11,707::task::588::TaskManager.Task::(_updateState) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::moving from state init -> state preparing
>Thread-1855::INFO::2012-12-09 12:45:11,708::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1855::DEBUG::2012-12-09 12:45:11,708::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5077f6a0-016b-4a70-bf88-d7e320af14de`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1855::DEBUG::2012-12-09 12:45:11,709::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1855::DEBUG::2012-12-09 12:45:11,709::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1855::DEBUG::2012-12-09 12:45:11,710::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5077f6a0-016b-4a70-bf88-d7e320af14de`::Granted request
>Thread-1855::DEBUG::2012-12-09 12:45:11,711::task::817::TaskManager.Task::(resourceAcquired) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1855::DEBUG::2012-12-09 12:45:11,712::task::978::TaskManager.Task::(_decref) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::ref 1 aborting False
>Thread-1855::DEBUG::2012-12-09 12:45:11,715::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1855::DEBUG::2012-12-09 12:45:11,717::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:11,766::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0407304 s, 25.1 MB/s\n'; <rc> = 0
>Thread-1855::DEBUG::2012-12-09 12:45:11,832::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1855::WARNING::2012-12-09 12:45:11,834::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1855::DEBUG::2012-12-09 12:45:11,834::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1855::WARNING::2012-12-09 12:45:11,835::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1855::INFO::2012-12-09 12:45:11,836::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1855::DEBUG::2012-12-09 12:45:11,836::task::1172::TaskManager.Task::(prepare) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1855::DEBUG::2012-12-09 12:45:11,837::task::588::TaskManager.Task::(_updateState) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::moving from state preparing -> state finished
>Thread-1855::DEBUG::2012-12-09 12:45:11,837::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1855::DEBUG::2012-12-09 12:45:11,837::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1855::DEBUG::2012-12-09 12:45:11,837::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1855::DEBUG::2012-12-09 12:45:11,838::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1855::DEBUG::2012-12-09 12:45:11,838::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1855::DEBUG::2012-12-09 12:45:11,838::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1855::DEBUG::2012-12-09 12:45:11,838::task::978::TaskManager.Task::(_decref) Task=`fd77d54c-f33c-46f1-b8d7-9f4b88c5174a`::ref 0 aborting False
>Thread-1856::DEBUG::2012-12-09 12:45:12,330::task::588::TaskManager.Task::(_updateState) Task=`90e8870f-4c27-41e5-a810-71f18ffddd28`::moving from state init -> state preparing
>Thread-1856::INFO::2012-12-09 12:45:12,331::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1856::INFO::2012-12-09 12:45:12,332::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153851509094', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1856::DEBUG::2012-12-09 12:45:12,333::task::1172::TaskManager.Task::(prepare) Task=`90e8870f-4c27-41e5-a810-71f18ffddd28`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0153851509094', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.7', 'code': 358, 'valid': False}}
>Thread-1856::DEBUG::2012-12-09 12:45:12,333::task::588::TaskManager.Task::(_updateState) Task=`90e8870f-4c27-41e5-a810-71f18ffddd28`::moving from state preparing -> state finished
>Thread-1856::DEBUG::2012-12-09 12:45:12,334::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1856::DEBUG::2012-12-09 12:45:12,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1856::DEBUG::2012-12-09 12:45:12,335::task::978::TaskManager.Task::(_decref) Task=`90e8870f-4c27-41e5-a810-71f18ffddd28`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:13,773::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:13,815::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018178 s, 56.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:15,822::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:15,862::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167652 s, 61.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:17,634::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:17,636::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:17,747::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:17,748::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:17,748::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:17,749::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:17,869::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:17,921::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267628 s, 38.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:19,928::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:19,996::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0287967 s, 35.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:20,418::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:20,433::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000874957 s, 4.7 MB/s\n'; <rc> = 0
>Thread-1862::DEBUG::2012-12-09 12:45:21,935::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1862::DEBUG::2012-12-09 12:45:21,936::task::588::TaskManager.Task::(_updateState) Task=`36659a9f-b830-4a6a-b2cc-82cfb1048cad`::moving from state init -> state preparing
>Thread-1862::INFO::2012-12-09 12:45:21,936::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1862::INFO::2012-12-09 12:45:21,937::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1862::DEBUG::2012-12-09 12:45:21,937::task::1172::TaskManager.Task::(prepare) Task=`36659a9f-b830-4a6a-b2cc-82cfb1048cad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1862::DEBUG::2012-12-09 12:45:21,937::task::588::TaskManager.Task::(_updateState) Task=`36659a9f-b830-4a6a-b2cc-82cfb1048cad`::moving from state preparing -> state finished
>Thread-1862::DEBUG::2012-12-09 12:45:21,937::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1862::DEBUG::2012-12-09 12:45:21,937::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1862::DEBUG::2012-12-09 12:45:21,938::task::978::TaskManager.Task::(_decref) Task=`36659a9f-b830-4a6a-b2cc-82cfb1048cad`::ref 0 aborting False
>Thread-1863::DEBUG::2012-12-09 12:45:21,951::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1863::DEBUG::2012-12-09 12:45:21,951::task::588::TaskManager.Task::(_updateState) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::moving from state init -> state preparing
>Thread-1863::INFO::2012-12-09 12:45:21,952::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1863::DEBUG::2012-12-09 12:45:21,952::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`04cd3b9b-a189-4ce0-911b-58f7dfaa02b0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1863::DEBUG::2012-12-09 12:45:21,952::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1863::DEBUG::2012-12-09 12:45:21,953::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1863::DEBUG::2012-12-09 12:45:21,953::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`04cd3b9b-a189-4ce0-911b-58f7dfaa02b0`::Granted request
>Thread-1863::DEBUG::2012-12-09 12:45:21,953::task::817::TaskManager.Task::(resourceAcquired) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1863::DEBUG::2012-12-09 12:45:21,954::task::978::TaskManager.Task::(_decref) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::ref 1 aborting False
>Thread-1863::DEBUG::2012-12-09 12:45:21,954::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1863::DEBUG::2012-12-09 12:45:21,955::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:22,002::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:22,040::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158642 s, 64.5 MB/s\n'; <rc> = 0
>Thread-1863::DEBUG::2012-12-09 12:45:22,053::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1863::WARNING::2012-12-09 12:45:22,054::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1863::DEBUG::2012-12-09 12:45:22,054::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1863::WARNING::2012-12-09 12:45:22,054::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1863::INFO::2012-12-09 12:45:22,055::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1863::DEBUG::2012-12-09 12:45:22,055::task::1172::TaskManager.Task::(prepare) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1863::DEBUG::2012-12-09 12:45:22,055::task::588::TaskManager.Task::(_updateState) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::moving from state preparing -> state finished
>Thread-1863::DEBUG::2012-12-09 12:45:22,055::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1863::DEBUG::2012-12-09 12:45:22,056::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1863::DEBUG::2012-12-09 12:45:22,056::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1863::DEBUG::2012-12-09 12:45:22,056::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1863::DEBUG::2012-12-09 12:45:22,056::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1863::DEBUG::2012-12-09 12:45:22,056::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1863::DEBUG::2012-12-09 12:45:22,057::task::978::TaskManager.Task::(_decref) Task=`6105e4c1-7f62-4db5-9950-81d2411f806d`::ref 0 aborting False
>Thread-1864::DEBUG::2012-12-09 12:45:22,888::task::588::TaskManager.Task::(_updateState) Task=`2a4d9ef1-1664-47ed-b0dc-fd1ccb804253`::moving from state init -> state preparing
>Thread-1864::INFO::2012-12-09 12:45:22,888::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1864::INFO::2012-12-09 12:45:22,889::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162808895111', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-1864::DEBUG::2012-12-09 12:45:22,889::task::1172::TaskManager.Task::(prepare) Task=`2a4d9ef1-1664-47ed-b0dc-fd1ccb804253`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162808895111', 'lastCheck': '2.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-1864::DEBUG::2012-12-09 12:45:22,889::task::588::TaskManager.Task::(_updateState) Task=`2a4d9ef1-1664-47ed-b0dc-fd1ccb804253`::moving from state preparing -> state finished
>Thread-1864::DEBUG::2012-12-09 12:45:22,890::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1864::DEBUG::2012-12-09 12:45:22,890::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1864::DEBUG::2012-12-09 12:45:22,890::task::978::TaskManager.Task::(_decref) Task=`2a4d9ef1-1664-47ed-b0dc-fd1ccb804253`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:24,047::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:24,089::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157595 s, 65.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:26,095::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:26,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226859 s, 45.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:27,750::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:27,751::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:27,871::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:27,873::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:27,873::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:27,874::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:28,150::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:28,202::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265206 s, 38.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:30,209::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:30,261::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234036 s, 43.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:30,438::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:30,450::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000494205 s, 8.3 MB/s\n'; <rc> = 0
>Thread-1870::DEBUG::2012-12-09 12:45:32,170::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1870::DEBUG::2012-12-09 12:45:32,171::task::588::TaskManager.Task::(_updateState) Task=`e854a743-5540-4823-b4d6-4b1eec66eeec`::moving from state init -> state preparing
>Thread-1870::INFO::2012-12-09 12:45:32,171::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1870::INFO::2012-12-09 12:45:32,171::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1870::DEBUG::2012-12-09 12:45:32,171::task::1172::TaskManager.Task::(prepare) Task=`e854a743-5540-4823-b4d6-4b1eec66eeec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1870::DEBUG::2012-12-09 12:45:32,172::task::588::TaskManager.Task::(_updateState) Task=`e854a743-5540-4823-b4d6-4b1eec66eeec`::moving from state preparing -> state finished
>Thread-1870::DEBUG::2012-12-09 12:45:32,172::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1870::DEBUG::2012-12-09 12:45:32,172::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1870::DEBUG::2012-12-09 12:45:32,172::task::978::TaskManager.Task::(_decref) Task=`e854a743-5540-4823-b4d6-4b1eec66eeec`::ref 0 aborting False
>Thread-1871::DEBUG::2012-12-09 12:45:32,189::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1871::DEBUG::2012-12-09 12:45:32,190::task::588::TaskManager.Task::(_updateState) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::moving from state init -> state preparing
>Thread-1871::INFO::2012-12-09 12:45:32,190::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1871::DEBUG::2012-12-09 12:45:32,190::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1ef2c881-d73e-4178-953d-1137e9eb58b8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1871::DEBUG::2012-12-09 12:45:32,191::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1871::DEBUG::2012-12-09 12:45:32,191::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1871::DEBUG::2012-12-09 12:45:32,191::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1ef2c881-d73e-4178-953d-1137e9eb58b8`::Granted request
>Thread-1871::DEBUG::2012-12-09 12:45:32,192::task::817::TaskManager.Task::(resourceAcquired) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1871::DEBUG::2012-12-09 12:45:32,192::task::978::TaskManager.Task::(_decref) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::ref 1 aborting False
>Thread-1871::DEBUG::2012-12-09 12:45:32,193::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1871::DEBUG::2012-12-09 12:45:32,193::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:32,267::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1871::DEBUG::2012-12-09 12:45:32,301::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1871::WARNING::2012-12-09 12:45:32,302::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1871::DEBUG::2012-12-09 12:45:32,302::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1871::WARNING::2012-12-09 12:45:32,303::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1871::INFO::2012-12-09 12:45:32,303::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1871::DEBUG::2012-12-09 12:45:32,303::task::1172::TaskManager.Task::(prepare) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1871::DEBUG::2012-12-09 12:45:32,304::task::588::TaskManager.Task::(_updateState) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::moving from state preparing -> state finished
>Thread-1871::DEBUG::2012-12-09 12:45:32,304::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1871::DEBUG::2012-12-09 12:45:32,304::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1871::DEBUG::2012-12-09 12:45:32,304::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1871::DEBUG::2012-12-09 12:45:32,305::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1871::DEBUG::2012-12-09 12:45:32,305::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1871::DEBUG::2012-12-09 12:45:32,305::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1871::DEBUG::2012-12-09 12:45:32,305::task::978::TaskManager.Task::(_decref) Task=`eb0168ff-f152-41a3-9037-397bb89a10ce`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:32,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179164 s, 57.2 MB/s\n'; <rc> = 0
>Thread-1872::DEBUG::2012-12-09 12:45:33,282::task::588::TaskManager.Task::(_updateState) Task=`8c84734d-04e9-4abb-9388-fccab0272877`::moving from state init -> state preparing
>Thread-1872::INFO::2012-12-09 12:45:33,282::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1872::INFO::2012-12-09 12:45:33,283::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126810073853', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1872::DEBUG::2012-12-09 12:45:33,284::task::1172::TaskManager.Task::(prepare) Task=`8c84734d-04e9-4abb-9388-fccab0272877`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126810073853', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.4', 'code': 358, 'valid': False}}
>Thread-1872::DEBUG::2012-12-09 12:45:33,284::task::588::TaskManager.Task::(_updateState) Task=`8c84734d-04e9-4abb-9388-fccab0272877`::moving from state preparing -> state finished
>Thread-1872::DEBUG::2012-12-09 12:45:33,284::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1872::DEBUG::2012-12-09 12:45:33,285::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1872::DEBUG::2012-12-09 12:45:33,285::task::978::TaskManager.Task::(_decref) Task=`8c84734d-04e9-4abb-9388-fccab0272877`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:34,321::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:34,367::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197615 s, 51.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:36,374::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:36,418::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019007 s, 53.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:37,876::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:37,876::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:37,987::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:37,989::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:37,989::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:37,990::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:38,425::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:38,463::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153071 s, 66.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:40,454::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:40,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000771029 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:40,469::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:40,513::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162042 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:42,520::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:42,565::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208789 s, 49.0 MB/s\n'; <rc> = 0
>Thread-1878::DEBUG::2012-12-09 12:45:43,011::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1878::DEBUG::2012-12-09 12:45:43,012::task::588::TaskManager.Task::(_updateState) Task=`25c7cf58-1121-4bbd-8356-9b3dc1c4a035`::moving from state init -> state preparing
>Thread-1878::INFO::2012-12-09 12:45:43,013::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1878::INFO::2012-12-09 12:45:43,013::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1878::DEBUG::2012-12-09 12:45:43,014::task::1172::TaskManager.Task::(prepare) Task=`25c7cf58-1121-4bbd-8356-9b3dc1c4a035`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1878::DEBUG::2012-12-09 12:45:43,014::task::588::TaskManager.Task::(_updateState) Task=`25c7cf58-1121-4bbd-8356-9b3dc1c4a035`::moving from state preparing -> state finished
>Thread-1878::DEBUG::2012-12-09 12:45:43,015::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1878::DEBUG::2012-12-09 12:45:43,015::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1878::DEBUG::2012-12-09 12:45:43,015::task::978::TaskManager.Task::(_decref) Task=`25c7cf58-1121-4bbd-8356-9b3dc1c4a035`::ref 0 aborting False
>Thread-1879::DEBUG::2012-12-09 12:45:43,028::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1879::DEBUG::2012-12-09 12:45:43,029::task::588::TaskManager.Task::(_updateState) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::moving from state init -> state preparing
>Thread-1879::INFO::2012-12-09 12:45:43,029::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1879::DEBUG::2012-12-09 12:45:43,030::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`594ec5a0-eca1-4989-a554-018029cd6c99`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1879::DEBUG::2012-12-09 12:45:43,031::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1879::DEBUG::2012-12-09 12:45:43,031::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1879::DEBUG::2012-12-09 12:45:43,031::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`594ec5a0-eca1-4989-a554-018029cd6c99`::Granted request
>Thread-1879::DEBUG::2012-12-09 12:45:43,032::task::817::TaskManager.Task::(resourceAcquired) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1879::DEBUG::2012-12-09 12:45:43,033::task::978::TaskManager.Task::(_decref) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::ref 1 aborting False
>Thread-1879::DEBUG::2012-12-09 12:45:43,034::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1879::DEBUG::2012-12-09 12:45:43,035::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1879::DEBUG::2012-12-09 12:45:43,143::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1879::WARNING::2012-12-09 12:45:43,145::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1879::DEBUG::2012-12-09 12:45:43,145::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1879::WARNING::2012-12-09 12:45:43,146::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1879::INFO::2012-12-09 12:45:43,147::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1879::DEBUG::2012-12-09 12:45:43,147::task::1172::TaskManager.Task::(prepare) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1879::DEBUG::2012-12-09 12:45:43,147::task::588::TaskManager.Task::(_updateState) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::moving from state preparing -> state finished
>Thread-1879::DEBUG::2012-12-09 12:45:43,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1879::DEBUG::2012-12-09 12:45:43,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1879::DEBUG::2012-12-09 12:45:43,148::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1879::DEBUG::2012-12-09 12:45:43,148::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1879::DEBUG::2012-12-09 12:45:43,148::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1879::DEBUG::2012-12-09 12:45:43,148::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1879::DEBUG::2012-12-09 12:45:43,149::task::978::TaskManager.Task::(_decref) Task=`de01c2e8-a65d-49d8-bbe7-d26ff82e44e8`::ref 0 aborting False
>Thread-1880::DEBUG::2012-12-09 12:45:44,193::task::588::TaskManager.Task::(_updateState) Task=`d7191213-73c4-41a5-9d78-a29a662dc59f`::moving from state init -> state preparing
>Thread-1880::INFO::2012-12-09 12:45:44,193::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1880::INFO::2012-12-09 12:45:44,194::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0147669315338', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-1880::DEBUG::2012-12-09 12:45:44,194::task::1172::TaskManager.Task::(prepare) Task=`d7191213-73c4-41a5-9d78-a29a662dc59f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0147669315338', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-1880::DEBUG::2012-12-09 12:45:44,194::task::588::TaskManager.Task::(_updateState) Task=`d7191213-73c4-41a5-9d78-a29a662dc59f`::moving from state preparing -> state finished
>Thread-1880::DEBUG::2012-12-09 12:45:44,195::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1880::DEBUG::2012-12-09 12:45:44,195::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1880::DEBUG::2012-12-09 12:45:44,195::task::978::TaskManager.Task::(_decref) Task=`d7191213-73c4-41a5-9d78-a29a662dc59f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:44,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:44,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163461 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:46,618::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:46,664::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157988 s, 64.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:47,991::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:47,992::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:48,112::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:48,113::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:48,113::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:48,113::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:48,673::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:48,720::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183624 s, 55.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:45:50,476::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:45:50,485::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000653528 s, 6.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:50,727::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:50,769::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164371 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:52,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:52,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176348 s, 58.1 MB/s\n'; <rc> = 0
>Thread-1886::DEBUG::2012-12-09 12:45:53,982::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1886::DEBUG::2012-12-09 12:45:53,982::task::588::TaskManager.Task::(_updateState) Task=`9a5af417-eff3-4333-a71c-045b39658750`::moving from state init -> state preparing
>Thread-1886::INFO::2012-12-09 12:45:53,982::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1886::INFO::2012-12-09 12:45:53,983::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1886::DEBUG::2012-12-09 12:45:53,983::task::1172::TaskManager.Task::(prepare) Task=`9a5af417-eff3-4333-a71c-045b39658750`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1886::DEBUG::2012-12-09 12:45:53,983::task::588::TaskManager.Task::(_updateState) Task=`9a5af417-eff3-4333-a71c-045b39658750`::moving from state preparing -> state finished
>Thread-1886::DEBUG::2012-12-09 12:45:53,983::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1886::DEBUG::2012-12-09 12:45:53,984::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1886::DEBUG::2012-12-09 12:45:53,984::task::978::TaskManager.Task::(_decref) Task=`9a5af417-eff3-4333-a71c-045b39658750`::ref 0 aborting False
>Thread-1887::DEBUG::2012-12-09 12:45:54,000::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1887::DEBUG::2012-12-09 12:45:54,000::task::588::TaskManager.Task::(_updateState) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::moving from state init -> state preparing
>Thread-1887::INFO::2012-12-09 12:45:54,000::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1887::DEBUG::2012-12-09 12:45:54,001::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b39c7547-195e-4373-b896-a04c8b317795`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1887::DEBUG::2012-12-09 12:45:54,001::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1887::DEBUG::2012-12-09 12:45:54,001::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1887::DEBUG::2012-12-09 12:45:54,002::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b39c7547-195e-4373-b896-a04c8b317795`::Granted request
>Thread-1887::DEBUG::2012-12-09 12:45:54,002::task::817::TaskManager.Task::(resourceAcquired) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1887::DEBUG::2012-12-09 12:45:54,002::task::978::TaskManager.Task::(_decref) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::ref 1 aborting False
>Thread-1887::DEBUG::2012-12-09 12:45:54,003::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1887::DEBUG::2012-12-09 12:45:54,004::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1887::DEBUG::2012-12-09 12:45:54,117::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1887::WARNING::2012-12-09 12:45:54,118::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1887::DEBUG::2012-12-09 12:45:54,119::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1887::WARNING::2012-12-09 12:45:54,119::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1887::INFO::2012-12-09 12:45:54,120::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1887::DEBUG::2012-12-09 12:45:54,120::task::1172::TaskManager.Task::(prepare) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1887::DEBUG::2012-12-09 12:45:54,120::task::588::TaskManager.Task::(_updateState) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::moving from state preparing -> state finished
>Thread-1887::DEBUG::2012-12-09 12:45:54,121::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1887::DEBUG::2012-12-09 12:45:54,121::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1887::DEBUG::2012-12-09 12:45:54,121::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1887::DEBUG::2012-12-09 12:45:54,122::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1887::DEBUG::2012-12-09 12:45:54,122::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1887::DEBUG::2012-12-09 12:45:54,122::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1887::DEBUG::2012-12-09 12:45:54,123::task::978::TaskManager.Task::(_decref) Task=`1f342e3c-b7d6-425f-89df-bbda78ae3662`::ref 0 aborting False
>Thread-1888::DEBUG::2012-12-09 12:45:54,697::task::588::TaskManager.Task::(_updateState) Task=`f6a82582-851e-4e39-9687-151afe6cff91`::moving from state init -> state preparing
>Thread-1888::INFO::2012-12-09 12:45:54,698::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1888::INFO::2012-12-09 12:45:54,698::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00968408584595', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-1888::DEBUG::2012-12-09 12:45:54,698::task::1172::TaskManager.Task::(prepare) Task=`f6a82582-851e-4e39-9687-151afe6cff91`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00968408584595', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-1888::DEBUG::2012-12-09 12:45:54,698::task::588::TaskManager.Task::(_updateState) Task=`f6a82582-851e-4e39-9687-151afe6cff91`::moving from state preparing -> state finished
>Thread-1888::DEBUG::2012-12-09 12:45:54,698::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1888::DEBUG::2012-12-09 12:45:54,699::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1888::DEBUG::2012-12-09 12:45:54,699::task::978::TaskManager.Task::(_decref) Task=`f6a82582-851e-4e39-9687-151afe6cff91`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:45:54,824::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:54,862::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155593 s, 65.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:45:56,869::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:56,922::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274152 s, 37.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:45:58,114::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:45:58,116::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:45:58,225::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:45:58,227::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:45:58,227::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:45:58,228::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:45:58,929::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:45:58,974::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02102 s, 48.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:00,489::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:00,500::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00046349 s, 8.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:00,981::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:01,020::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161123 s, 63.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:03,026::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:03,068::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182587 s, 56.1 MB/s\n'; <rc> = 0
>Thread-1894::DEBUG::2012-12-09 12:46:04,181::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1894::DEBUG::2012-12-09 12:46:04,182::task::588::TaskManager.Task::(_updateState) Task=`5d0f5385-00a5-49e9-b44a-74810eb8f3a6`::moving from state init -> state preparing
>Thread-1894::INFO::2012-12-09 12:46:04,183::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1894::INFO::2012-12-09 12:46:04,184::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1894::DEBUG::2012-12-09 12:46:04,184::task::1172::TaskManager.Task::(prepare) Task=`5d0f5385-00a5-49e9-b44a-74810eb8f3a6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1894::DEBUG::2012-12-09 12:46:04,185::task::588::TaskManager.Task::(_updateState) Task=`5d0f5385-00a5-49e9-b44a-74810eb8f3a6`::moving from state preparing -> state finished
>Thread-1894::DEBUG::2012-12-09 12:46:04,185::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1894::DEBUG::2012-12-09 12:46:04,186::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1894::DEBUG::2012-12-09 12:46:04,186::task::978::TaskManager.Task::(_decref) Task=`5d0f5385-00a5-49e9-b44a-74810eb8f3a6`::ref 0 aborting False
>Thread-1895::DEBUG::2012-12-09 12:46:04,202::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1895::DEBUG::2012-12-09 12:46:04,203::task::588::TaskManager.Task::(_updateState) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::moving from state init -> state preparing
>Thread-1895::INFO::2012-12-09 12:46:04,204::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1895::DEBUG::2012-12-09 12:46:04,204::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`19d384e0-3e1b-4d20-8a95-c92065403927`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1895::DEBUG::2012-12-09 12:46:04,205::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1895::DEBUG::2012-12-09 12:46:04,205::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1895::DEBUG::2012-12-09 12:46:04,206::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`19d384e0-3e1b-4d20-8a95-c92065403927`::Granted request
>Thread-1895::DEBUG::2012-12-09 12:46:04,206::task::817::TaskManager.Task::(resourceAcquired) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1895::DEBUG::2012-12-09 12:46:04,207::task::978::TaskManager.Task::(_decref) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::ref 1 aborting False
>Thread-1895::DEBUG::2012-12-09 12:46:04,208::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1895::DEBUG::2012-12-09 12:46:04,209::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1895::DEBUG::2012-12-09 12:46:04,328::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1895::WARNING::2012-12-09 12:46:04,330::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1895::DEBUG::2012-12-09 12:46:04,331::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1895::WARNING::2012-12-09 12:46:04,332::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1895::INFO::2012-12-09 12:46:04,333::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1895::DEBUG::2012-12-09 12:46:04,333::task::1172::TaskManager.Task::(prepare) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1895::DEBUG::2012-12-09 12:46:04,334::task::588::TaskManager.Task::(_updateState) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::moving from state preparing -> state finished
>Thread-1895::DEBUG::2012-12-09 12:46:04,334::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1895::DEBUG::2012-12-09 12:46:04,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1895::DEBUG::2012-12-09 12:46:04,335::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1895::DEBUG::2012-12-09 12:46:04,336::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1895::DEBUG::2012-12-09 12:46:04,336::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1895::DEBUG::2012-12-09 12:46:04,337::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1895::DEBUG::2012-12-09 12:46:04,337::task::978::TaskManager.Task::(_decref) Task=`6d49217b-8138-4d99-8cd3-6e67fdb8c150`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:05,075::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:05,122::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238543 s, 42.9 MB/s\n'; <rc> = 0
>Thread-1896::DEBUG::2012-12-09 12:46:05,143::task::588::TaskManager.Task::(_updateState) Task=`58783b4a-6016-45a8-8eb6-ad8982d2cc52`::moving from state init -> state preparing
>Thread-1896::INFO::2012-12-09 12:46:05,144::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1896::INFO::2012-12-09 12:46:05,144::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116050243378', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1896::DEBUG::2012-12-09 12:46:05,145::task::1172::TaskManager.Task::(prepare) Task=`58783b4a-6016-45a8-8eb6-ad8982d2cc52`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116050243378', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1896::DEBUG::2012-12-09 12:46:05,145::task::588::TaskManager.Task::(_updateState) Task=`58783b4a-6016-45a8-8eb6-ad8982d2cc52`::moving from state preparing -> state finished
>Thread-1896::DEBUG::2012-12-09 12:46:05,146::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1896::DEBUG::2012-12-09 12:46:05,146::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1896::DEBUG::2012-12-09 12:46:05,146::task::978::TaskManager.Task::(_decref) Task=`58783b4a-6016-45a8-8eb6-ad8982d2cc52`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:07,130::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:07,169::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160946 s, 63.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:46:08,228::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:08,229::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:08,325::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:08,327::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:08,327::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:08,327::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:46:09,175::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:09,214::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159326 s, 64.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:10,502::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:10,510::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000458776 s, 8.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:11,221::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:11,266::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196112 s, 52.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:13,273::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:13,314::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168166 s, 60.9 MB/s\n'; <rc> = 0
>Thread-1902::DEBUG::2012-12-09 12:46:14,835::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1902::DEBUG::2012-12-09 12:46:14,836::task::588::TaskManager.Task::(_updateState) Task=`11ed49b6-7ed6-442c-a04b-e8d5af40a593`::moving from state init -> state preparing
>Thread-1902::INFO::2012-12-09 12:46:14,836::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1902::INFO::2012-12-09 12:46:14,837::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1902::DEBUG::2012-12-09 12:46:14,837::task::1172::TaskManager.Task::(prepare) Task=`11ed49b6-7ed6-442c-a04b-e8d5af40a593`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1902::DEBUG::2012-12-09 12:46:14,837::task::588::TaskManager.Task::(_updateState) Task=`11ed49b6-7ed6-442c-a04b-e8d5af40a593`::moving from state preparing -> state finished
>Thread-1902::DEBUG::2012-12-09 12:46:14,837::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1902::DEBUG::2012-12-09 12:46:14,837::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1902::DEBUG::2012-12-09 12:46:14,838::task::978::TaskManager.Task::(_decref) Task=`11ed49b6-7ed6-442c-a04b-e8d5af40a593`::ref 0 aborting False
>Thread-1903::DEBUG::2012-12-09 12:46:14,851::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1903::DEBUG::2012-12-09 12:46:14,851::task::588::TaskManager.Task::(_updateState) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::moving from state init -> state preparing
>Thread-1903::INFO::2012-12-09 12:46:14,851::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1903::DEBUG::2012-12-09 12:46:14,852::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`202d1349-6b48-45c7-a5f1-f15edecc462f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1903::DEBUG::2012-12-09 12:46:14,852::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1903::DEBUG::2012-12-09 12:46:14,852::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1903::DEBUG::2012-12-09 12:46:14,852::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`202d1349-6b48-45c7-a5f1-f15edecc462f`::Granted request
>Thread-1903::DEBUG::2012-12-09 12:46:14,853::task::817::TaskManager.Task::(resourceAcquired) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1903::DEBUG::2012-12-09 12:46:14,853::task::978::TaskManager.Task::(_decref) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::ref 1 aborting False
>Thread-1903::DEBUG::2012-12-09 12:46:14,853::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1903::DEBUG::2012-12-09 12:46:14,854::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1903::DEBUG::2012-12-09 12:46:14,953::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1903::WARNING::2012-12-09 12:46:14,955::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1903::DEBUG::2012-12-09 12:46:14,956::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1903::WARNING::2012-12-09 12:46:14,957::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1903::INFO::2012-12-09 12:46:14,957::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1903::DEBUG::2012-12-09 12:46:14,957::task::1172::TaskManager.Task::(prepare) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1903::DEBUG::2012-12-09 12:46:14,958::task::588::TaskManager.Task::(_updateState) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::moving from state preparing -> state finished
>Thread-1903::DEBUG::2012-12-09 12:46:14,958::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1903::DEBUG::2012-12-09 12:46:14,958::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1903::DEBUG::2012-12-09 12:46:14,958::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1903::DEBUG::2012-12-09 12:46:14,959::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1903::DEBUG::2012-12-09 12:46:14,959::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1903::DEBUG::2012-12-09 12:46:14,959::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1903::DEBUG::2012-12-09 12:46:14,959::task::978::TaskManager.Task::(_decref) Task=`c281b6d5-ed95-44ef-9b8a-a1f81cb18281`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:15,320::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:15,367::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202627 s, 50.5 MB/s\n'; <rc> = 0
>Thread-1904::DEBUG::2012-12-09 12:46:15,644::task::588::TaskManager.Task::(_updateState) Task=`418e9390-d2f6-4768-a29a-9efb6d5cd31b`::moving from state init -> state preparing
>Thread-1904::INFO::2012-12-09 12:46:15,645::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1904::INFO::2012-12-09 12:46:15,646::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00828981399536', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1904::DEBUG::2012-12-09 12:46:15,646::task::1172::TaskManager.Task::(prepare) Task=`418e9390-d2f6-4768-a29a-9efb6d5cd31b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00828981399536', 'lastCheck': '5.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-1904::DEBUG::2012-12-09 12:46:15,647::task::588::TaskManager.Task::(_updateState) Task=`418e9390-d2f6-4768-a29a-9efb6d5cd31b`::moving from state preparing -> state finished
>Thread-1904::DEBUG::2012-12-09 12:46:15,647::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1904::DEBUG::2012-12-09 12:46:15,647::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1904::DEBUG::2012-12-09 12:46:15,648::task::978::TaskManager.Task::(_decref) Task=`418e9390-d2f6-4768-a29a-9efb6d5cd31b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:17,373::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:17,414::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176485 s, 58.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:46:18,328::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:18,329::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:18,427::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:18,428::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:18,428::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:18,429::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:46:19,421::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:19,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219382 s, 46.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:20,513::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:20,525::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000768625 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:21,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:21,520::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206278 s, 49.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:23,526::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:23,574::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226054 s, 45.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:25,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:25,627::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220207 s, 46.5 MB/s\n'; <rc> = 0
>Thread-1910::DEBUG::2012-12-09 12:46:25,725::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1910::DEBUG::2012-12-09 12:46:25,726::task::588::TaskManager.Task::(_updateState) Task=`74cffa0e-9364-4916-93e5-c6caad6ecf49`::moving from state init -> state preparing
>Thread-1910::INFO::2012-12-09 12:46:25,726::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1910::INFO::2012-12-09 12:46:25,727::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1910::DEBUG::2012-12-09 12:46:25,728::task::1172::TaskManager.Task::(prepare) Task=`74cffa0e-9364-4916-93e5-c6caad6ecf49`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1910::DEBUG::2012-12-09 12:46:25,728::task::588::TaskManager.Task::(_updateState) Task=`74cffa0e-9364-4916-93e5-c6caad6ecf49`::moving from state preparing -> state finished
>Thread-1910::DEBUG::2012-12-09 12:46:25,728::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1910::DEBUG::2012-12-09 12:46:25,729::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1910::DEBUG::2012-12-09 12:46:25,729::task::978::TaskManager.Task::(_decref) Task=`74cffa0e-9364-4916-93e5-c6caad6ecf49`::ref 0 aborting False
>Thread-1911::DEBUG::2012-12-09 12:46:25,745::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1911::DEBUG::2012-12-09 12:46:25,746::task::588::TaskManager.Task::(_updateState) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::moving from state init -> state preparing
>Thread-1911::INFO::2012-12-09 12:46:25,746::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1911::DEBUG::2012-12-09 12:46:25,746::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dc5e49bd-82c0-46fc-b499-c6f996428e92`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1911::DEBUG::2012-12-09 12:46:25,746::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1911::DEBUG::2012-12-09 12:46:25,747::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1911::DEBUG::2012-12-09 12:46:25,747::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`dc5e49bd-82c0-46fc-b499-c6f996428e92`::Granted request
>Thread-1911::DEBUG::2012-12-09 12:46:25,747::task::817::TaskManager.Task::(resourceAcquired) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1911::DEBUG::2012-12-09 12:46:25,747::task::978::TaskManager.Task::(_decref) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::ref 1 aborting False
>Thread-1911::DEBUG::2012-12-09 12:46:25,748::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1911::DEBUG::2012-12-09 12:46:25,749::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1911::DEBUG::2012-12-09 12:46:25,849::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1911::WARNING::2012-12-09 12:46:25,851::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1911::DEBUG::2012-12-09 12:46:25,851::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1911::WARNING::2012-12-09 12:46:25,851::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1911::INFO::2012-12-09 12:46:25,852::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1911::DEBUG::2012-12-09 12:46:25,852::task::1172::TaskManager.Task::(prepare) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1911::DEBUG::2012-12-09 12:46:25,852::task::588::TaskManager.Task::(_updateState) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::moving from state preparing -> state finished
>Thread-1911::DEBUG::2012-12-09 12:46:25,852::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1911::DEBUG::2012-12-09 12:46:25,852::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1911::DEBUG::2012-12-09 12:46:25,853::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1911::DEBUG::2012-12-09 12:46:25,853::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1911::DEBUG::2012-12-09 12:46:25,853::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1911::DEBUG::2012-12-09 12:46:25,853::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1911::DEBUG::2012-12-09 12:46:25,854::task::978::TaskManager.Task::(_decref) Task=`3637df80-a1bc-4f15-a724-6bca77bbfb7d`::ref 0 aborting False
>Thread-1912::DEBUG::2012-12-09 12:46:27,318::task::588::TaskManager.Task::(_updateState) Task=`9417f786-94c4-4b58-8747-2db8215b23b7`::moving from state init -> state preparing
>Thread-1912::INFO::2012-12-09 12:46:27,319::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1912::INFO::2012-12-09 12:46:27,320::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126068592072', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-1912::DEBUG::2012-12-09 12:46:27,320::task::1172::TaskManager.Task::(prepare) Task=`9417f786-94c4-4b58-8747-2db8215b23b7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126068592072', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-1912::DEBUG::2012-12-09 12:46:27,320::task::588::TaskManager.Task::(_updateState) Task=`9417f786-94c4-4b58-8747-2db8215b23b7`::moving from state preparing -> state finished
>Thread-1912::DEBUG::2012-12-09 12:46:27,321::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1912::DEBUG::2012-12-09 12:46:27,321::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1912::DEBUG::2012-12-09 12:46:27,322::task::978::TaskManager.Task::(_decref) Task=`9417f786-94c4-4b58-8747-2db8215b23b7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:27,634::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:27,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016916 s, 60.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:46:28,430::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:28,431::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:28,546::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:28,547::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:28,548::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:28,548::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:46:29,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:29,731::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250283 s, 40.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:30,529::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:30,542::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000835417 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:31,738::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:31,781::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168691 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:33,789::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:33,833::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193821 s, 52.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:35,840::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:35,887::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227602 s, 45.0 MB/s\n'; <rc> = 0
>Thread-1918::DEBUG::2012-12-09 12:46:36,415::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1918::DEBUG::2012-12-09 12:46:36,416::task::588::TaskManager.Task::(_updateState) Task=`597e0f2d-a7ee-45eb-abc8-14cf1c82ddd9`::moving from state init -> state preparing
>Thread-1918::INFO::2012-12-09 12:46:36,416::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1918::INFO::2012-12-09 12:46:36,416::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1918::DEBUG::2012-12-09 12:46:36,417::task::1172::TaskManager.Task::(prepare) Task=`597e0f2d-a7ee-45eb-abc8-14cf1c82ddd9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1918::DEBUG::2012-12-09 12:46:36,417::task::588::TaskManager.Task::(_updateState) Task=`597e0f2d-a7ee-45eb-abc8-14cf1c82ddd9`::moving from state preparing -> state finished
>Thread-1918::DEBUG::2012-12-09 12:46:36,417::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1918::DEBUG::2012-12-09 12:46:36,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1918::DEBUG::2012-12-09 12:46:36,417::task::978::TaskManager.Task::(_decref) Task=`597e0f2d-a7ee-45eb-abc8-14cf1c82ddd9`::ref 0 aborting False
>Thread-1919::DEBUG::2012-12-09 12:46:36,431::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1919::DEBUG::2012-12-09 12:46:36,432::task::588::TaskManager.Task::(_updateState) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::moving from state init -> state preparing
>Thread-1919::INFO::2012-12-09 12:46:36,432::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1919::DEBUG::2012-12-09 12:46:36,433::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`584ae48e-203e-4fba-90f6-b938637571ca`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1919::DEBUG::2012-12-09 12:46:36,433::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1919::DEBUG::2012-12-09 12:46:36,434::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1919::DEBUG::2012-12-09 12:46:36,434::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`584ae48e-203e-4fba-90f6-b938637571ca`::Granted request
>Thread-1919::DEBUG::2012-12-09 12:46:36,435::task::817::TaskManager.Task::(resourceAcquired) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1919::DEBUG::2012-12-09 12:46:36,435::task::978::TaskManager.Task::(_decref) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::ref 1 aborting False
>Thread-1919::DEBUG::2012-12-09 12:46:36,437::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1919::DEBUG::2012-12-09 12:46:36,438::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1919::DEBUG::2012-12-09 12:46:36,546::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1919::WARNING::2012-12-09 12:46:36,547::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1919::DEBUG::2012-12-09 12:46:36,548::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1919::WARNING::2012-12-09 12:46:36,548::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1919::INFO::2012-12-09 12:46:36,548::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1919::DEBUG::2012-12-09 12:46:36,549::task::1172::TaskManager.Task::(prepare) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1919::DEBUG::2012-12-09 12:46:36,549::task::588::TaskManager.Task::(_updateState) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::moving from state preparing -> state finished
>Thread-1919::DEBUG::2012-12-09 12:46:36,549::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1919::DEBUG::2012-12-09 12:46:36,549::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1919::DEBUG::2012-12-09 12:46:36,549::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1919::DEBUG::2012-12-09 12:46:36,550::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1919::DEBUG::2012-12-09 12:46:36,550::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1919::DEBUG::2012-12-09 12:46:36,550::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1919::DEBUG::2012-12-09 12:46:36,550::task::978::TaskManager.Task::(_decref) Task=`232c81ab-e257-435c-a21a-a9c12afc4c07`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:37,894::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1920::DEBUG::2012-12-09 12:46:37,900::task::588::TaskManager.Task::(_updateState) Task=`12ad7109-8925-4d99-b8e9-3baa2d7e7b49`::moving from state init -> state preparing
>Thread-1920::INFO::2012-12-09 12:46:37,901::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1920::INFO::2012-12-09 12:46:37,902::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135159492493', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-1920::DEBUG::2012-12-09 12:46:37,902::task::1172::TaskManager.Task::(prepare) Task=`12ad7109-8925-4d99-b8e9-3baa2d7e7b49`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135159492493', 'lastCheck': '7.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-1920::DEBUG::2012-12-09 12:46:37,903::task::588::TaskManager.Task::(_updateState) Task=`12ad7109-8925-4d99-b8e9-3baa2d7e7b49`::moving from state preparing -> state finished
>Thread-1920::DEBUG::2012-12-09 12:46:37,904::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1920::DEBUG::2012-12-09 12:46:37,905::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1920::DEBUG::2012-12-09 12:46:37,905::task::978::TaskManager.Task::(_decref) Task=`12ad7109-8925-4d99-b8e9-3baa2d7e7b49`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:37,955::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.033401 s, 30.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:46:38,549::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:38,550::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:38,655::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:38,656::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:38,656::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:38,657::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:46:39,962::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:40,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162836 s, 62.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:40,545::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:40,558::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000663373 s, 6.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:42,008::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:42,055::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221809 s, 46.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:44,062::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:44,119::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171751 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:46,126::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:46,174::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230167 s, 44.5 MB/s\n'; <rc> = 0
>Thread-1926::DEBUG::2012-12-09 12:46:47,413::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1926::DEBUG::2012-12-09 12:46:47,414::task::588::TaskManager.Task::(_updateState) Task=`82e6a6e4-06aa-45c0-87c1-0b6ff8814ef2`::moving from state init -> state preparing
>Thread-1926::INFO::2012-12-09 12:46:47,414::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1926::INFO::2012-12-09 12:46:47,415::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1926::DEBUG::2012-12-09 12:46:47,415::task::1172::TaskManager.Task::(prepare) Task=`82e6a6e4-06aa-45c0-87c1-0b6ff8814ef2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1926::DEBUG::2012-12-09 12:46:47,415::task::588::TaskManager.Task::(_updateState) Task=`82e6a6e4-06aa-45c0-87c1-0b6ff8814ef2`::moving from state preparing -> state finished
>Thread-1926::DEBUG::2012-12-09 12:46:47,415::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1926::DEBUG::2012-12-09 12:46:47,415::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1926::DEBUG::2012-12-09 12:46:47,415::task::978::TaskManager.Task::(_decref) Task=`82e6a6e4-06aa-45c0-87c1-0b6ff8814ef2`::ref 0 aborting False
>Thread-1927::DEBUG::2012-12-09 12:46:47,427::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1927::DEBUG::2012-12-09 12:46:47,428::task::588::TaskManager.Task::(_updateState) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::moving from state init -> state preparing
>Thread-1927::INFO::2012-12-09 12:46:47,428::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1927::DEBUG::2012-12-09 12:46:47,428::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c6ddaf0-39c1-43ae-8f9d-33ed6c79f74a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1927::DEBUG::2012-12-09 12:46:47,428::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1927::DEBUG::2012-12-09 12:46:47,429::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1927::DEBUG::2012-12-09 12:46:47,429::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c6ddaf0-39c1-43ae-8f9d-33ed6c79f74a`::Granted request
>Thread-1927::DEBUG::2012-12-09 12:46:47,429::task::817::TaskManager.Task::(resourceAcquired) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1927::DEBUG::2012-12-09 12:46:47,429::task::978::TaskManager.Task::(_decref) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::ref 1 aborting False
>Thread-1927::DEBUG::2012-12-09 12:46:47,430::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1927::DEBUG::2012-12-09 12:46:47,430::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1927::DEBUG::2012-12-09 12:46:47,529::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1927::WARNING::2012-12-09 12:46:47,530::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1927::DEBUG::2012-12-09 12:46:47,530::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1927::WARNING::2012-12-09 12:46:47,531::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1927::INFO::2012-12-09 12:46:47,531::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1927::DEBUG::2012-12-09 12:46:47,531::task::1172::TaskManager.Task::(prepare) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1927::DEBUG::2012-12-09 12:46:47,531::task::588::TaskManager.Task::(_updateState) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::moving from state preparing -> state finished
>Thread-1927::DEBUG::2012-12-09 12:46:47,532::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1927::DEBUG::2012-12-09 12:46:47,532::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1927::DEBUG::2012-12-09 12:46:47,532::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1927::DEBUG::2012-12-09 12:46:47,532::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1927::DEBUG::2012-12-09 12:46:47,532::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1927::DEBUG::2012-12-09 12:46:47,533::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1927::DEBUG::2012-12-09 12:46:47,533::task::978::TaskManager.Task::(_decref) Task=`5870ced6-a47f-4748-9860-31aef773faa0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:48,181::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:48,223::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156367 s, 65.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:46:48,658::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:48,659::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:48,761::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:48,762::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:48,763::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:48,763::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1928::DEBUG::2012-12-09 12:46:48,794::task::588::TaskManager.Task::(_updateState) Task=`d808beec-7002-41da-95a4-ea8cecb8c337`::moving from state init -> state preparing
>Thread-1928::INFO::2012-12-09 12:46:48,794::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1928::INFO::2012-12-09 12:46:48,795::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134871006012', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-1928::DEBUG::2012-12-09 12:46:48,795::task::1172::TaskManager.Task::(prepare) Task=`d808beec-7002-41da-95a4-ea8cecb8c337`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134871006012', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-1928::DEBUG::2012-12-09 12:46:48,795::task::588::TaskManager.Task::(_updateState) Task=`d808beec-7002-41da-95a4-ea8cecb8c337`::moving from state preparing -> state finished
>Thread-1928::DEBUG::2012-12-09 12:46:48,795::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1928::DEBUG::2012-12-09 12:46:48,796::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1928::DEBUG::2012-12-09 12:46:48,796::task::978::TaskManager.Task::(_decref) Task=`d808beec-7002-41da-95a4-ea8cecb8c337`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:46:50,229::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:50,280::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208165 s, 49.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:46:50,561::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:46:50,576::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000935133 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:52,286::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:52,331::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017178 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:54,337::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:54,377::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166283 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:56,384::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:56,424::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016423 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:46:58,430::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1934::DEBUG::2012-12-09 12:46:58,457::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1934::DEBUG::2012-12-09 12:46:58,459::task::588::TaskManager.Task::(_updateState) Task=`db500a14-021a-43e7-9eb6-4e56e57f4b57`::moving from state init -> state preparing
>Thread-1934::INFO::2012-12-09 12:46:58,460::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1934::INFO::2012-12-09 12:46:58,461::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1934::DEBUG::2012-12-09 12:46:58,461::task::1172::TaskManager.Task::(prepare) Task=`db500a14-021a-43e7-9eb6-4e56e57f4b57`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1934::DEBUG::2012-12-09 12:46:58,462::task::588::TaskManager.Task::(_updateState) Task=`db500a14-021a-43e7-9eb6-4e56e57f4b57`::moving from state preparing -> state finished
>Thread-1934::DEBUG::2012-12-09 12:46:58,463::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1934::DEBUG::2012-12-09 12:46:58,463::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1934::DEBUG::2012-12-09 12:46:58,464::task::978::TaskManager.Task::(_decref) Task=`db500a14-021a-43e7-9eb6-4e56e57f4b57`::ref 0 aborting False
>Thread-1935::DEBUG::2012-12-09 12:46:58,483::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1935::DEBUG::2012-12-09 12:46:58,485::task::588::TaskManager.Task::(_updateState) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::moving from state init -> state preparing
>Thread-1935::INFO::2012-12-09 12:46:58,486::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1935::DEBUG::2012-12-09 12:46:58,487::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25143b3b-0b93-4e0b-ba33-27d2a59b09d8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1935::DEBUG::2012-12-09 12:46:58,488::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1935::DEBUG::2012-12-09 12:46:58,490::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1935::DEBUG::2012-12-09 12:46:58,490::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`25143b3b-0b93-4e0b-ba33-27d2a59b09d8`::Granted request
>Thread-1935::DEBUG::2012-12-09 12:46:58,492::task::817::TaskManager.Task::(resourceAcquired) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1935::DEBUG::2012-12-09 12:46:58,493::task::978::TaskManager.Task::(_decref) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::ref 1 aborting False
>Thread-1935::DEBUG::2012-12-09 12:46:58,497::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1935::DEBUG::2012-12-09 12:46:58,499::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:46:58,534::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0370732 s, 27.6 MB/s\n'; <rc> = 0
>Thread-1935::DEBUG::2012-12-09 12:46:58,612::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1935::WARNING::2012-12-09 12:46:58,614::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1935::DEBUG::2012-12-09 12:46:58,614::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1935::WARNING::2012-12-09 12:46:58,615::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1935::INFO::2012-12-09 12:46:58,616::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1935::DEBUG::2012-12-09 12:46:58,616::task::1172::TaskManager.Task::(prepare) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1935::DEBUG::2012-12-09 12:46:58,617::task::588::TaskManager.Task::(_updateState) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::moving from state preparing -> state finished
>Thread-1935::DEBUG::2012-12-09 12:46:58,617::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1935::DEBUG::2012-12-09 12:46:58,617::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1935::DEBUG::2012-12-09 12:46:58,618::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1935::DEBUG::2012-12-09 12:46:58,618::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1935::DEBUG::2012-12-09 12:46:58,618::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1935::DEBUG::2012-12-09 12:46:58,618::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1935::DEBUG::2012-12-09 12:46:58,618::task::978::TaskManager.Task::(_decref) Task=`f600b68e-9a43-44ae-a390-a2d666df88be`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:46:58,764::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:46:58,765::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:46:58,864::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:46:58,866::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:46:58,866::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:46:58,866::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1936::DEBUG::2012-12-09 12:46:59,649::task::588::TaskManager.Task::(_updateState) Task=`7e283d97-9c9d-4b8b-824e-d711e51ca8df`::moving from state init -> state preparing
>Thread-1936::INFO::2012-12-09 12:46:59,649::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1936::INFO::2012-12-09 12:46:59,650::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160069465637', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-1936::DEBUG::2012-12-09 12:46:59,650::task::1172::TaskManager.Task::(prepare) Task=`7e283d97-9c9d-4b8b-824e-d711e51ca8df`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160069465637', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-1936::DEBUG::2012-12-09 12:46:59,650::task::588::TaskManager.Task::(_updateState) Task=`7e283d97-9c9d-4b8b-824e-d711e51ca8df`::moving from state preparing -> state finished
>Thread-1936::DEBUG::2012-12-09 12:46:59,650::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1936::DEBUG::2012-12-09 12:46:59,650::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1936::DEBUG::2012-12-09 12:46:59,651::task::978::TaskManager.Task::(_decref) Task=`7e283d97-9c9d-4b8b-824e-d711e51ca8df`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:00,543::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:00,581::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:00,602::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000420985 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:00,614::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187151 s, 54.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:02,620::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:02,660::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167877 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:04,666::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:04,706::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171069 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:06,713::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:06,762::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264777 s, 38.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:08,768::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:08,821::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281557 s, 36.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:08,867::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:08,869::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:47:08,976::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:08,977::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:08,978::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:08,978::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1942::DEBUG::2012-12-09 12:47:09,531::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1942::DEBUG::2012-12-09 12:47:09,532::task::588::TaskManager.Task::(_updateState) Task=`d60d5433-ac9c-43e5-9971-dfbfdc4c8669`::moving from state init -> state preparing
>Thread-1942::INFO::2012-12-09 12:47:09,533::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1942::INFO::2012-12-09 12:47:09,533::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1942::DEBUG::2012-12-09 12:47:09,533::task::1172::TaskManager.Task::(prepare) Task=`d60d5433-ac9c-43e5-9971-dfbfdc4c8669`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1942::DEBUG::2012-12-09 12:47:09,533::task::588::TaskManager.Task::(_updateState) Task=`d60d5433-ac9c-43e5-9971-dfbfdc4c8669`::moving from state preparing -> state finished
>Thread-1942::DEBUG::2012-12-09 12:47:09,533::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1942::DEBUG::2012-12-09 12:47:09,534::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1942::DEBUG::2012-12-09 12:47:09,534::task::978::TaskManager.Task::(_decref) Task=`d60d5433-ac9c-43e5-9971-dfbfdc4c8669`::ref 0 aborting False
>Thread-1943::DEBUG::2012-12-09 12:47:09,549::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1943::DEBUG::2012-12-09 12:47:09,550::task::588::TaskManager.Task::(_updateState) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::moving from state init -> state preparing
>Thread-1943::INFO::2012-12-09 12:47:09,550::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1943::DEBUG::2012-12-09 12:47:09,551::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8e2b2f8b-7549-423e-b81c-447c8d58d4e8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1943::DEBUG::2012-12-09 12:47:09,551::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1943::DEBUG::2012-12-09 12:47:09,552::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1943::DEBUG::2012-12-09 12:47:09,552::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8e2b2f8b-7549-423e-b81c-447c8d58d4e8`::Granted request
>Thread-1943::DEBUG::2012-12-09 12:47:09,553::task::817::TaskManager.Task::(resourceAcquired) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1943::DEBUG::2012-12-09 12:47:09,553::task::978::TaskManager.Task::(_decref) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::ref 1 aborting False
>Thread-1943::DEBUG::2012-12-09 12:47:09,555::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1943::DEBUG::2012-12-09 12:47:09,556::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1943::DEBUG::2012-12-09 12:47:09,656::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1943::WARNING::2012-12-09 12:47:09,657::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1943::DEBUG::2012-12-09 12:47:09,657::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1943::WARNING::2012-12-09 12:47:09,658::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1943::INFO::2012-12-09 12:47:09,658::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1943::DEBUG::2012-12-09 12:47:09,658::task::1172::TaskManager.Task::(prepare) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1943::DEBUG::2012-12-09 12:47:09,659::task::588::TaskManager.Task::(_updateState) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::moving from state preparing -> state finished
>Thread-1943::DEBUG::2012-12-09 12:47:09,659::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1943::DEBUG::2012-12-09 12:47:09,659::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1943::DEBUG::2012-12-09 12:47:09,659::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1943::DEBUG::2012-12-09 12:47:09,659::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1943::DEBUG::2012-12-09 12:47:09,660::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1943::DEBUG::2012-12-09 12:47:09,660::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1943::DEBUG::2012-12-09 12:47:09,660::task::978::TaskManager.Task::(_decref) Task=`bc619290-8f3c-4d0a-b08f-3dd9e25373d0`::ref 0 aborting False
>Thread-1944::DEBUG::2012-12-09 12:47:10,419::task::588::TaskManager.Task::(_updateState) Task=`d483b658-a04c-4e7f-a41b-ca2de5aeb6b1`::moving from state init -> state preparing
>Thread-1944::INFO::2012-12-09 12:47:10,419::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1944::INFO::2012-12-09 12:47:10,419::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.022999048233', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-1944::DEBUG::2012-12-09 12:47:10,420::task::1172::TaskManager.Task::(prepare) Task=`d483b658-a04c-4e7f-a41b-ca2de5aeb6b1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.022999048233', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-1944::DEBUG::2012-12-09 12:47:10,420::task::588::TaskManager.Task::(_updateState) Task=`d483b658-a04c-4e7f-a41b-ca2de5aeb6b1`::moving from state preparing -> state finished
>Thread-1944::DEBUG::2012-12-09 12:47:10,420::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1944::DEBUG::2012-12-09 12:47:10,420::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1944::DEBUG::2012-12-09 12:47:10,420::task::978::TaskManager.Task::(_decref) Task=`d483b658-a04c-4e7f-a41b-ca2de5aeb6b1`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:47:10,613::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:10,622::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000461535 s, 8.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:10,828::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:10,866::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168453 s, 60.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:12,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:12,912::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163316 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:14,919::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:14,965::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212618 s, 48.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:16,971::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:17,015::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195582 s, 52.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:18,979::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:18,980::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:19,022::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:19,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192784 s, 53.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:19,080::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:19,081::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:19,081::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:19,081::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1950::DEBUG::2012-12-09 12:47:20,219::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1950::DEBUG::2012-12-09 12:47:20,219::task::588::TaskManager.Task::(_updateState) Task=`a6849395-575b-4eef-8d6d-fb90cc3d3846`::moving from state init -> state preparing
>Thread-1950::INFO::2012-12-09 12:47:20,219::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1950::INFO::2012-12-09 12:47:20,220::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1950::DEBUG::2012-12-09 12:47:20,220::task::1172::TaskManager.Task::(prepare) Task=`a6849395-575b-4eef-8d6d-fb90cc3d3846`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1950::DEBUG::2012-12-09 12:47:20,220::task::588::TaskManager.Task::(_updateState) Task=`a6849395-575b-4eef-8d6d-fb90cc3d3846`::moving from state preparing -> state finished
>Thread-1950::DEBUG::2012-12-09 12:47:20,220::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1950::DEBUG::2012-12-09 12:47:20,220::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1950::DEBUG::2012-12-09 12:47:20,221::task::978::TaskManager.Task::(_decref) Task=`a6849395-575b-4eef-8d6d-fb90cc3d3846`::ref 0 aborting False
>Thread-1951::DEBUG::2012-12-09 12:47:20,233::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1951::DEBUG::2012-12-09 12:47:20,233::task::588::TaskManager.Task::(_updateState) Task=`13b81494-cee4-499b-8526-162e679b37eb`::moving from state init -> state preparing
>Thread-1951::INFO::2012-12-09 12:47:20,234::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1951::DEBUG::2012-12-09 12:47:20,234::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ea81fc21-cfca-44ce-adcf-30dda86797ed`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1951::DEBUG::2012-12-09 12:47:20,234::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1951::DEBUG::2012-12-09 12:47:20,234::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1951::DEBUG::2012-12-09 12:47:20,235::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ea81fc21-cfca-44ce-adcf-30dda86797ed`::Granted request
>Thread-1951::DEBUG::2012-12-09 12:47:20,235::task::817::TaskManager.Task::(resourceAcquired) Task=`13b81494-cee4-499b-8526-162e679b37eb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1951::DEBUG::2012-12-09 12:47:20,235::task::978::TaskManager.Task::(_decref) Task=`13b81494-cee4-499b-8526-162e679b37eb`::ref 1 aborting False
>Thread-1951::DEBUG::2012-12-09 12:47:20,236::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1951::DEBUG::2012-12-09 12:47:20,236::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1951::DEBUG::2012-12-09 12:47:20,340::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1951::WARNING::2012-12-09 12:47:20,341::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1951::DEBUG::2012-12-09 12:47:20,341::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1951::WARNING::2012-12-09 12:47:20,341::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1951::INFO::2012-12-09 12:47:20,342::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1951::DEBUG::2012-12-09 12:47:20,342::task::1172::TaskManager.Task::(prepare) Task=`13b81494-cee4-499b-8526-162e679b37eb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1951::DEBUG::2012-12-09 12:47:20,342::task::588::TaskManager.Task::(_updateState) Task=`13b81494-cee4-499b-8526-162e679b37eb`::moving from state preparing -> state finished
>Thread-1951::DEBUG::2012-12-09 12:47:20,342::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1951::DEBUG::2012-12-09 12:47:20,343::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1951::DEBUG::2012-12-09 12:47:20,343::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1951::DEBUG::2012-12-09 12:47:20,343::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1951::DEBUG::2012-12-09 12:47:20,343::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1951::DEBUG::2012-12-09 12:47:20,343::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1951::DEBUG::2012-12-09 12:47:20,344::task::978::TaskManager.Task::(_decref) Task=`13b81494-cee4-499b-8526-162e679b37eb`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:47:20,624::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:20,635::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00084508 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:21,071::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:21,116::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199628 s, 51.3 MB/s\n'; <rc> = 0
>Thread-1952::DEBUG::2012-12-09 12:47:21,822::task::588::TaskManager.Task::(_updateState) Task=`a810693d-d3e8-43eb-bddd-cc68d5629609`::moving from state init -> state preparing
>Thread-1952::INFO::2012-12-09 12:47:21,822::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1952::INFO::2012-12-09 12:47:21,823::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114350318909', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1952::DEBUG::2012-12-09 12:47:21,823::task::1172::TaskManager.Task::(prepare) Task=`a810693d-d3e8-43eb-bddd-cc68d5629609`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114350318909', 'lastCheck': '1.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-1952::DEBUG::2012-12-09 12:47:21,824::task::588::TaskManager.Task::(_updateState) Task=`a810693d-d3e8-43eb-bddd-cc68d5629609`::moving from state preparing -> state finished
>Thread-1952::DEBUG::2012-12-09 12:47:21,824::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1952::DEBUG::2012-12-09 12:47:21,825::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1952::DEBUG::2012-12-09 12:47:21,825::task::978::TaskManager.Task::(_decref) Task=`a810693d-d3e8-43eb-bddd-cc68d5629609`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:23,123::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:23,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198262 s, 51.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:25,175::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:25,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195907 s, 52.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:27,225::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:27,270::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176561 s, 58.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:29,082::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:29,083::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:47:29,186::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:29,188::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:29,188::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:29,189::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:47:29,277::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:29,323::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227749 s, 45.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:47:30,639::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:30,652::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000733926 s, 5.6 MB/s\n'; <rc> = 0
>Thread-1958::DEBUG::2012-12-09 12:47:31,023::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1958::DEBUG::2012-12-09 12:47:31,024::task::588::TaskManager.Task::(_updateState) Task=`0e7babfe-9d19-43ca-9b68-c43daa2457b1`::moving from state init -> state preparing
>Thread-1958::INFO::2012-12-09 12:47:31,024::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1958::INFO::2012-12-09 12:47:31,024::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1958::DEBUG::2012-12-09 12:47:31,024::task::1172::TaskManager.Task::(prepare) Task=`0e7babfe-9d19-43ca-9b68-c43daa2457b1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1958::DEBUG::2012-12-09 12:47:31,025::task::588::TaskManager.Task::(_updateState) Task=`0e7babfe-9d19-43ca-9b68-c43daa2457b1`::moving from state preparing -> state finished
>Thread-1958::DEBUG::2012-12-09 12:47:31,025::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1958::DEBUG::2012-12-09 12:47:31,025::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1958::DEBUG::2012-12-09 12:47:31,025::task::978::TaskManager.Task::(_decref) Task=`0e7babfe-9d19-43ca-9b68-c43daa2457b1`::ref 0 aborting False
>Thread-1959::DEBUG::2012-12-09 12:47:31,043::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1959::DEBUG::2012-12-09 12:47:31,044::task::588::TaskManager.Task::(_updateState) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::moving from state init -> state preparing
>Thread-1959::INFO::2012-12-09 12:47:31,044::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1959::DEBUG::2012-12-09 12:47:31,045::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0fc7d284-cb84-41aa-be36-226392ce9809`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1959::DEBUG::2012-12-09 12:47:31,046::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1959::DEBUG::2012-12-09 12:47:31,046::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1959::DEBUG::2012-12-09 12:47:31,047::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0fc7d284-cb84-41aa-be36-226392ce9809`::Granted request
>Thread-1959::DEBUG::2012-12-09 12:47:31,047::task::817::TaskManager.Task::(resourceAcquired) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1959::DEBUG::2012-12-09 12:47:31,047::task::978::TaskManager.Task::(_decref) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::ref 1 aborting False
>Thread-1959::DEBUG::2012-12-09 12:47:31,048::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1959::DEBUG::2012-12-09 12:47:31,048::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1959::DEBUG::2012-12-09 12:47:31,156::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1959::WARNING::2012-12-09 12:47:31,158::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1959::DEBUG::2012-12-09 12:47:31,158::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1959::WARNING::2012-12-09 12:47:31,158::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1959::INFO::2012-12-09 12:47:31,159::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1959::DEBUG::2012-12-09 12:47:31,159::task::1172::TaskManager.Task::(prepare) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1959::DEBUG::2012-12-09 12:47:31,159::task::588::TaskManager.Task::(_updateState) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::moving from state preparing -> state finished
>Thread-1959::DEBUG::2012-12-09 12:47:31,159::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1959::DEBUG::2012-12-09 12:47:31,159::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1959::DEBUG::2012-12-09 12:47:31,160::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1959::DEBUG::2012-12-09 12:47:31,160::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1959::DEBUG::2012-12-09 12:47:31,160::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1959::DEBUG::2012-12-09 12:47:31,160::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1959::DEBUG::2012-12-09 12:47:31,161::task::978::TaskManager.Task::(_decref) Task=`693863a1-8ed8-41cd-a6a5-c3a0ca37777b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:31,329::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:31,380::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257353 s, 39.8 MB/s\n'; <rc> = 0
>Thread-1960::DEBUG::2012-12-09 12:47:32,341::task::588::TaskManager.Task::(_updateState) Task=`6f3bb05b-5f86-4152-a9a0-3682e4bab749`::moving from state init -> state preparing
>Thread-1960::INFO::2012-12-09 12:47:32,341::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1960::INFO::2012-12-09 12:47:32,342::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0144989490509', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-1960::DEBUG::2012-12-09 12:47:32,343::task::1172::TaskManager.Task::(prepare) Task=`6f3bb05b-5f86-4152-a9a0-3682e4bab749`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0144989490509', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-1960::DEBUG::2012-12-09 12:47:32,343::task::588::TaskManager.Task::(_updateState) Task=`6f3bb05b-5f86-4152-a9a0-3682e4bab749`::moving from state preparing -> state finished
>Thread-1960::DEBUG::2012-12-09 12:47:32,343::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1960::DEBUG::2012-12-09 12:47:32,344::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1960::DEBUG::2012-12-09 12:47:32,344::task::978::TaskManager.Task::(_decref) Task=`6f3bb05b-5f86-4152-a9a0-3682e4bab749`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:33,387::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:33,434::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021255 s, 48.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:35,441::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:35,482::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167439 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:37,491::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:37,535::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161604 s, 63.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:39,190::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:39,190::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:47:39,288::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:39,289::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:39,289::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:39,290::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:47:39,542::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:39,580::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162687 s, 62.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:47:40,658::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:40,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000364079 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:41,586::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:41,626::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165191 s, 62.0 MB/s\n'; <rc> = 0
>Thread-1966::DEBUG::2012-12-09 12:47:41,805::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1966::DEBUG::2012-12-09 12:47:41,805::task::588::TaskManager.Task::(_updateState) Task=`f762dd34-ca82-42da-963c-1077ed765e74`::moving from state init -> state preparing
>Thread-1966::INFO::2012-12-09 12:47:41,806::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1966::INFO::2012-12-09 12:47:41,806::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1966::DEBUG::2012-12-09 12:47:41,806::task::1172::TaskManager.Task::(prepare) Task=`f762dd34-ca82-42da-963c-1077ed765e74`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1966::DEBUG::2012-12-09 12:47:41,807::task::588::TaskManager.Task::(_updateState) Task=`f762dd34-ca82-42da-963c-1077ed765e74`::moving from state preparing -> state finished
>Thread-1966::DEBUG::2012-12-09 12:47:41,807::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1966::DEBUG::2012-12-09 12:47:41,807::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1966::DEBUG::2012-12-09 12:47:41,807::task::978::TaskManager.Task::(_decref) Task=`f762dd34-ca82-42da-963c-1077ed765e74`::ref 0 aborting False
>Thread-1967::DEBUG::2012-12-09 12:47:41,820::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1967::DEBUG::2012-12-09 12:47:41,821::task::588::TaskManager.Task::(_updateState) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::moving from state init -> state preparing
>Thread-1967::INFO::2012-12-09 12:47:41,821::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1967::DEBUG::2012-12-09 12:47:41,822::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`550e1692-34d7-4cb4-aa00-a2f3b271960c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1967::DEBUG::2012-12-09 12:47:41,822::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1967::DEBUG::2012-12-09 12:47:41,822::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1967::DEBUG::2012-12-09 12:47:41,823::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`550e1692-34d7-4cb4-aa00-a2f3b271960c`::Granted request
>Thread-1967::DEBUG::2012-12-09 12:47:41,823::task::817::TaskManager.Task::(resourceAcquired) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1967::DEBUG::2012-12-09 12:47:41,824::task::978::TaskManager.Task::(_decref) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::ref 1 aborting False
>Thread-1967::DEBUG::2012-12-09 12:47:41,824::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1967::DEBUG::2012-12-09 12:47:41,825::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1967::DEBUG::2012-12-09 12:47:41,925::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1967::WARNING::2012-12-09 12:47:41,926::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1967::DEBUG::2012-12-09 12:47:41,926::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1967::WARNING::2012-12-09 12:47:41,927::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1967::INFO::2012-12-09 12:47:41,927::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1967::DEBUG::2012-12-09 12:47:41,927::task::1172::TaskManager.Task::(prepare) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1967::DEBUG::2012-12-09 12:47:41,928::task::588::TaskManager.Task::(_updateState) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::moving from state preparing -> state finished
>Thread-1967::DEBUG::2012-12-09 12:47:41,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1967::DEBUG::2012-12-09 12:47:41,928::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1967::DEBUG::2012-12-09 12:47:41,928::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1967::DEBUG::2012-12-09 12:47:41,929::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1967::DEBUG::2012-12-09 12:47:41,929::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1967::DEBUG::2012-12-09 12:47:41,929::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1967::DEBUG::2012-12-09 12:47:41,929::task::978::TaskManager.Task::(_decref) Task=`75cf816b-3b87-4018-be9a-072608055a5b`::ref 0 aborting False
>Thread-1968::DEBUG::2012-12-09 12:47:42,960::task::588::TaskManager.Task::(_updateState) Task=`764219fd-5d02-468a-8199-16567103372f`::moving from state init -> state preparing
>Thread-1968::INFO::2012-12-09 12:47:42,960::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1968::INFO::2012-12-09 12:47:42,961::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124061107635', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.7', 'code': 358, 'valid': False}}
>Thread-1968::DEBUG::2012-12-09 12:47:42,961::task::1172::TaskManager.Task::(prepare) Task=`764219fd-5d02-468a-8199-16567103372f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124061107635', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.7', 'code': 358, 'valid': False}}
>Thread-1968::DEBUG::2012-12-09 12:47:42,961::task::588::TaskManager.Task::(_updateState) Task=`764219fd-5d02-468a-8199-16567103372f`::moving from state preparing -> state finished
>Thread-1968::DEBUG::2012-12-09 12:47:42,961::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1968::DEBUG::2012-12-09 12:47:42,961::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1968::DEBUG::2012-12-09 12:47:42,962::task::978::TaskManager.Task::(_decref) Task=`764219fd-5d02-468a-8199-16567103372f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:43,633::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:43,673::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167149 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:45,680::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:45,719::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162932 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:47,725::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:47,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187633 s, 54.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:49,290::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:49,291::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:47:49,415::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:49,417::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:49,417::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:49,418::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:47:49,776::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:49,820::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188074 s, 54.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:47:50,673::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:47:50,681::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000540869 s, 7.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:51,827::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:51,867::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170787 s, 60.0 MB/s\n'; <rc> = 0
>Thread-1974::DEBUG::2012-12-09 12:47:52,808::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1974::DEBUG::2012-12-09 12:47:52,809::task::588::TaskManager.Task::(_updateState) Task=`c8aee36d-e213-4b11-9a8c-db48f2f0686e`::moving from state init -> state preparing
>Thread-1974::INFO::2012-12-09 12:47:52,809::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1974::INFO::2012-12-09 12:47:52,810::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1974::DEBUG::2012-12-09 12:47:52,810::task::1172::TaskManager.Task::(prepare) Task=`c8aee36d-e213-4b11-9a8c-db48f2f0686e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1974::DEBUG::2012-12-09 12:47:52,811::task::588::TaskManager.Task::(_updateState) Task=`c8aee36d-e213-4b11-9a8c-db48f2f0686e`::moving from state preparing -> state finished
>Thread-1974::DEBUG::2012-12-09 12:47:52,811::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1974::DEBUG::2012-12-09 12:47:52,812::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1974::DEBUG::2012-12-09 12:47:52,812::task::978::TaskManager.Task::(_decref) Task=`c8aee36d-e213-4b11-9a8c-db48f2f0686e`::ref 0 aborting False
>Thread-1975::DEBUG::2012-12-09 12:47:52,831::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1975::DEBUG::2012-12-09 12:47:52,831::task::588::TaskManager.Task::(_updateState) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::moving from state init -> state preparing
>Thread-1975::INFO::2012-12-09 12:47:52,831::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1975::DEBUG::2012-12-09 12:47:52,832::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f92cdf0-d709-46f9-bedc-cac70d6820d5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1975::DEBUG::2012-12-09 12:47:52,832::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1975::DEBUG::2012-12-09 12:47:52,832::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1975::DEBUG::2012-12-09 12:47:52,832::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f92cdf0-d709-46f9-bedc-cac70d6820d5`::Granted request
>Thread-1975::DEBUG::2012-12-09 12:47:52,833::task::817::TaskManager.Task::(resourceAcquired) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1975::DEBUG::2012-12-09 12:47:52,833::task::978::TaskManager.Task::(_decref) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::ref 1 aborting False
>Thread-1975::DEBUG::2012-12-09 12:47:52,834::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1975::DEBUG::2012-12-09 12:47:52,834::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1975::DEBUG::2012-12-09 12:47:52,938::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1975::WARNING::2012-12-09 12:47:52,939::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1975::DEBUG::2012-12-09 12:47:52,939::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1975::WARNING::2012-12-09 12:47:52,940::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1975::INFO::2012-12-09 12:47:52,940::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1975::DEBUG::2012-12-09 12:47:52,941::task::1172::TaskManager.Task::(prepare) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1975::DEBUG::2012-12-09 12:47:52,941::task::588::TaskManager.Task::(_updateState) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::moving from state preparing -> state finished
>Thread-1975::DEBUG::2012-12-09 12:47:52,941::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1975::DEBUG::2012-12-09 12:47:52,941::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1975::DEBUG::2012-12-09 12:47:52,941::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1975::DEBUG::2012-12-09 12:47:52,942::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1975::DEBUG::2012-12-09 12:47:52,942::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1975::DEBUG::2012-12-09 12:47:52,942::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1975::DEBUG::2012-12-09 12:47:52,942::task::978::TaskManager.Task::(_decref) Task=`7e5985b5-7354-44c7-989e-2e7eed861ff0`::ref 0 aborting False
>Thread-1976::DEBUG::2012-12-09 12:47:53,489::task::588::TaskManager.Task::(_updateState) Task=`5f4c70a5-981c-4e2d-b2d3-549af001d4dc`::moving from state init -> state preparing
>Thread-1976::INFO::2012-12-09 12:47:53,490::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1976::INFO::2012-12-09 12:47:53,490::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00841808319092', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1976::DEBUG::2012-12-09 12:47:53,490::task::1172::TaskManager.Task::(prepare) Task=`5f4c70a5-981c-4e2d-b2d3-549af001d4dc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00841808319092', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.1', 'code': 358, 'valid': False}}
>Thread-1976::DEBUG::2012-12-09 12:47:53,490::task::588::TaskManager.Task::(_updateState) Task=`5f4c70a5-981c-4e2d-b2d3-549af001d4dc`::moving from state preparing -> state finished
>Thread-1976::DEBUG::2012-12-09 12:47:53,491::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1976::DEBUG::2012-12-09 12:47:53,491::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1976::DEBUG::2012-12-09 12:47:53,491::task::978::TaskManager.Task::(_decref) Task=`5f4c70a5-981c-4e2d-b2d3-549af001d4dc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:47:53,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:53,915::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183404 s, 55.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:55,921::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:55,961::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166154 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:47:57,968::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:47:58,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171232 s, 59.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:47:59,419::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:47:59,420::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:47:59,525::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:47:59,527::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:47:59,527::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:47:59,528::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:48:00,015::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:00,063::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233898 s, 43.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:48:00,685::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:00,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000932837 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:02,070::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:02,108::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157346 s, 65.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:04,115::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:04,163::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245143 s, 41.8 MB/s\n'; <rc> = 0
>Thread-1982::DEBUG::2012-12-09 12:48:04,606::task::588::TaskManager.Task::(_updateState) Task=`4de0a740-f7ab-4635-89c1-b2cf4598add5`::moving from state init -> state preparing
>Thread-1982::INFO::2012-12-09 12:48:04,607::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1982::INFO::2012-12-09 12:48:04,607::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143349170685', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-1982::DEBUG::2012-12-09 12:48:04,607::task::1172::TaskManager.Task::(prepare) Task=`4de0a740-f7ab-4635-89c1-b2cf4598add5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143349170685', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.1', 'code': 358, 'valid': False}}
>Thread-1982::DEBUG::2012-12-09 12:48:04,607::task::588::TaskManager.Task::(_updateState) Task=`4de0a740-f7ab-4635-89c1-b2cf4598add5`::moving from state preparing -> state finished
>Thread-1982::DEBUG::2012-12-09 12:48:04,607::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1982::DEBUG::2012-12-09 12:48:04,608::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1982::DEBUG::2012-12-09 12:48:04,608::task::978::TaskManager.Task::(_decref) Task=`4de0a740-f7ab-4635-89c1-b2cf4598add5`::ref 0 aborting False
>Thread-1984::DEBUG::2012-12-09 12:48:04,961::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1984::DEBUG::2012-12-09 12:48:04,962::task::588::TaskManager.Task::(_updateState) Task=`9ef78954-1e0e-4814-a423-6077713f0895`::moving from state init -> state preparing
>Thread-1984::INFO::2012-12-09 12:48:04,962::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1984::INFO::2012-12-09 12:48:04,963::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1984::DEBUG::2012-12-09 12:48:04,963::task::1172::TaskManager.Task::(prepare) Task=`9ef78954-1e0e-4814-a423-6077713f0895`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1984::DEBUG::2012-12-09 12:48:04,963::task::588::TaskManager.Task::(_updateState) Task=`9ef78954-1e0e-4814-a423-6077713f0895`::moving from state preparing -> state finished
>Thread-1984::DEBUG::2012-12-09 12:48:04,963::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1984::DEBUG::2012-12-09 12:48:04,963::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1984::DEBUG::2012-12-09 12:48:04,964::task::978::TaskManager.Task::(_decref) Task=`9ef78954-1e0e-4814-a423-6077713f0895`::ref 0 aborting False
>Thread-1985::DEBUG::2012-12-09 12:48:04,977::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1985::DEBUG::2012-12-09 12:48:04,977::task::588::TaskManager.Task::(_updateState) Task=`ffd63313-912e-4229-8678-99249c8b2485`::moving from state init -> state preparing
>Thread-1985::INFO::2012-12-09 12:48:04,978::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1985::DEBUG::2012-12-09 12:48:04,978::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`404e4e8e-46b8-43b6-baa9-a26a2bc99996`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1985::DEBUG::2012-12-09 12:48:04,978::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1985::DEBUG::2012-12-09 12:48:04,978::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1985::DEBUG::2012-12-09 12:48:04,979::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`404e4e8e-46b8-43b6-baa9-a26a2bc99996`::Granted request
>Thread-1985::DEBUG::2012-12-09 12:48:04,979::task::817::TaskManager.Task::(resourceAcquired) Task=`ffd63313-912e-4229-8678-99249c8b2485`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1985::DEBUG::2012-12-09 12:48:04,979::task::978::TaskManager.Task::(_decref) Task=`ffd63313-912e-4229-8678-99249c8b2485`::ref 1 aborting False
>Thread-1985::DEBUG::2012-12-09 12:48:04,980::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1985::DEBUG::2012-12-09 12:48:04,980::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1985::DEBUG::2012-12-09 12:48:05,077::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1985::WARNING::2012-12-09 12:48:05,078::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1985::DEBUG::2012-12-09 12:48:05,078::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1985::WARNING::2012-12-09 12:48:05,078::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1985::INFO::2012-12-09 12:48:05,079::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1985::DEBUG::2012-12-09 12:48:05,079::task::1172::TaskManager.Task::(prepare) Task=`ffd63313-912e-4229-8678-99249c8b2485`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1985::DEBUG::2012-12-09 12:48:05,079::task::588::TaskManager.Task::(_updateState) Task=`ffd63313-912e-4229-8678-99249c8b2485`::moving from state preparing -> state finished
>Thread-1985::DEBUG::2012-12-09 12:48:05,079::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1985::DEBUG::2012-12-09 12:48:05,080::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1985::DEBUG::2012-12-09 12:48:05,080::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1985::DEBUG::2012-12-09 12:48:05,080::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1985::DEBUG::2012-12-09 12:48:05,080::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1985::DEBUG::2012-12-09 12:48:05,081::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1985::DEBUG::2012-12-09 12:48:05,081::task::978::TaskManager.Task::(_decref) Task=`ffd63313-912e-4229-8678-99249c8b2485`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:48:06,170::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:06,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199325 s, 51.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:08,227::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:08,273::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212214 s, 48.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:48:09,529::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:48:09,530::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:48:09,635::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:48:09,636::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:48:09,636::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:48:09,637::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:48:10,280::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:10,326::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208534 s, 49.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:48:10,702::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:10,717::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00085714 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:12,333::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:12,374::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163469 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:14,381::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:14,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167227 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:16,428::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:16,476::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.022409 s, 45.7 MB/s\n'; <rc> = 0
>Thread-1990::DEBUG::2012-12-09 12:48:16,510::task::588::TaskManager.Task::(_updateState) Task=`15aeccd0-ded6-4553-8562-e3fc214d1888`::moving from state init -> state preparing
>Thread-1990::INFO::2012-12-09 12:48:16,510::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1990::INFO::2012-12-09 12:48:16,511::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160281658173', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1990::DEBUG::2012-12-09 12:48:16,511::task::1172::TaskManager.Task::(prepare) Task=`15aeccd0-ded6-4553-8562-e3fc214d1888`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160281658173', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-1990::DEBUG::2012-12-09 12:48:16,512::task::588::TaskManager.Task::(_updateState) Task=`15aeccd0-ded6-4553-8562-e3fc214d1888`::moving from state preparing -> state finished
>Thread-1990::DEBUG::2012-12-09 12:48:16,512::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1990::DEBUG::2012-12-09 12:48:16,513::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1990::DEBUG::2012-12-09 12:48:16,513::task::978::TaskManager.Task::(_decref) Task=`15aeccd0-ded6-4553-8562-e3fc214d1888`::ref 0 aborting False
>Thread-1992::DEBUG::2012-12-09 12:48:16,962::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1992::DEBUG::2012-12-09 12:48:16,963::task::588::TaskManager.Task::(_updateState) Task=`335ab50e-c961-4f54-ba34-acf941b3065e`::moving from state init -> state preparing
>Thread-1992::INFO::2012-12-09 12:48:16,963::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1992::INFO::2012-12-09 12:48:16,964::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1992::DEBUG::2012-12-09 12:48:16,964::task::1172::TaskManager.Task::(prepare) Task=`335ab50e-c961-4f54-ba34-acf941b3065e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-1992::DEBUG::2012-12-09 12:48:16,965::task::588::TaskManager.Task::(_updateState) Task=`335ab50e-c961-4f54-ba34-acf941b3065e`::moving from state preparing -> state finished
>Thread-1992::DEBUG::2012-12-09 12:48:16,965::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1992::DEBUG::2012-12-09 12:48:16,966::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1992::DEBUG::2012-12-09 12:48:16,966::task::978::TaskManager.Task::(_decref) Task=`335ab50e-c961-4f54-ba34-acf941b3065e`::ref 0 aborting False
>Thread-1993::DEBUG::2012-12-09 12:48:16,989::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-1993::DEBUG::2012-12-09 12:48:16,990::task::588::TaskManager.Task::(_updateState) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::moving from state init -> state preparing
>Thread-1993::INFO::2012-12-09 12:48:16,991::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-1993::DEBUG::2012-12-09 12:48:16,992::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bc550cfb-9916-4627-aadf-896bf9a0a2a3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-1993::DEBUG::2012-12-09 12:48:16,992::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-1993::DEBUG::2012-12-09 12:48:16,993::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-1993::DEBUG::2012-12-09 12:48:16,993::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bc550cfb-9916-4627-aadf-896bf9a0a2a3`::Granted request
>Thread-1993::DEBUG::2012-12-09 12:48:16,994::task::817::TaskManager.Task::(resourceAcquired) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-1993::DEBUG::2012-12-09 12:48:16,994::task::978::TaskManager.Task::(_decref) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::ref 1 aborting False
>Thread-1993::DEBUG::2012-12-09 12:48:16,995::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-1993::DEBUG::2012-12-09 12:48:16,996::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-1993::DEBUG::2012-12-09 12:48:17,109::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-1993::WARNING::2012-12-09 12:48:17,110::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-1993::DEBUG::2012-12-09 12:48:17,110::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1993::WARNING::2012-12-09 12:48:17,110::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-1993::INFO::2012-12-09 12:48:17,111::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1993::DEBUG::2012-12-09 12:48:17,111::task::1172::TaskManager.Task::(prepare) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-1993::DEBUG::2012-12-09 12:48:17,111::task::588::TaskManager.Task::(_updateState) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::moving from state preparing -> state finished
>Thread-1993::DEBUG::2012-12-09 12:48:17,111::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-1993::DEBUG::2012-12-09 12:48:17,112::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1993::DEBUG::2012-12-09 12:48:17,112::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-1993::DEBUG::2012-12-09 12:48:17,112::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-1993::DEBUG::2012-12-09 12:48:17,112::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-1993::DEBUG::2012-12-09 12:48:17,112::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-1993::DEBUG::2012-12-09 12:48:17,113::task::978::TaskManager.Task::(_decref) Task=`60480dde-d24a-4eed-af80-dbc4b1958fe9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:48:18,484::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:18,527::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189235 s, 54.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:48:19,638::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:48:19,638::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:48:19,734::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:48:19,735::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:48:19,736::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:48:19,736::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:48:20,533::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:20,575::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180842 s, 56.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:48:20,720::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:20,728::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00039614 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:22,584::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:22,636::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246922 s, 41.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:24,643::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:24,692::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241851 s, 42.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:26,698::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:26,744::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211478 s, 48.4 MB/s\n'; <rc> = 0
>Thread-1998::DEBUG::2012-12-09 12:48:27,173::task::588::TaskManager.Task::(_updateState) Task=`d68664e0-9d90-450f-8101-856f45f7f34f`::moving from state init -> state preparing
>Thread-1998::INFO::2012-12-09 12:48:27,173::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-1998::INFO::2012-12-09 12:48:27,174::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00779891014099', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-1998::DEBUG::2012-12-09 12:48:27,174::task::1172::TaskManager.Task::(prepare) Task=`d68664e0-9d90-450f-8101-856f45f7f34f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00779891014099', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-1998::DEBUG::2012-12-09 12:48:27,174::task::588::TaskManager.Task::(_updateState) Task=`d68664e0-9d90-450f-8101-856f45f7f34f`::moving from state preparing -> state finished
>Thread-1998::DEBUG::2012-12-09 12:48:27,174::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-1998::DEBUG::2012-12-09 12:48:27,174::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-1998::DEBUG::2012-12-09 12:48:27,174::task::978::TaskManager.Task::(_decref) Task=`d68664e0-9d90-450f-8101-856f45f7f34f`::ref 0 aborting False
>Thread-2000::DEBUG::2012-12-09 12:48:27,608::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2000::DEBUG::2012-12-09 12:48:27,608::task::588::TaskManager.Task::(_updateState) Task=`c9c0d996-185c-4acc-8585-954136c57aec`::moving from state init -> state preparing
>Thread-2000::INFO::2012-12-09 12:48:27,608::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2000::INFO::2012-12-09 12:48:27,609::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2000::DEBUG::2012-12-09 12:48:27,609::task::1172::TaskManager.Task::(prepare) Task=`c9c0d996-185c-4acc-8585-954136c57aec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2000::DEBUG::2012-12-09 12:48:27,609::task::588::TaskManager.Task::(_updateState) Task=`c9c0d996-185c-4acc-8585-954136c57aec`::moving from state preparing -> state finished
>Thread-2000::DEBUG::2012-12-09 12:48:27,609::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2000::DEBUG::2012-12-09 12:48:27,609::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2000::DEBUG::2012-12-09 12:48:27,610::task::978::TaskManager.Task::(_decref) Task=`c9c0d996-185c-4acc-8585-954136c57aec`::ref 0 aborting False
>Thread-2001::DEBUG::2012-12-09 12:48:27,622::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2001::DEBUG::2012-12-09 12:48:27,623::task::588::TaskManager.Task::(_updateState) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::moving from state init -> state preparing
>Thread-2001::INFO::2012-12-09 12:48:27,623::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2001::DEBUG::2012-12-09 12:48:27,623::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`319b94fb-0b2f-4720-902a-0d6e6c5b0906`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2001::DEBUG::2012-12-09 12:48:27,624::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2001::DEBUG::2012-12-09 12:48:27,624::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2001::DEBUG::2012-12-09 12:48:27,624::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`319b94fb-0b2f-4720-902a-0d6e6c5b0906`::Granted request
>Thread-2001::DEBUG::2012-12-09 12:48:27,624::task::817::TaskManager.Task::(resourceAcquired) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2001::DEBUG::2012-12-09 12:48:27,625::task::978::TaskManager.Task::(_decref) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::ref 1 aborting False
>Thread-2001::DEBUG::2012-12-09 12:48:27,625::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2001::DEBUG::2012-12-09 12:48:27,626::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2001::DEBUG::2012-12-09 12:48:27,720::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2001::WARNING::2012-12-09 12:48:27,721::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2001::DEBUG::2012-12-09 12:48:27,722::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2001::WARNING::2012-12-09 12:48:27,722::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2001::INFO::2012-12-09 12:48:27,723::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2001::DEBUG::2012-12-09 12:48:27,723::task::1172::TaskManager.Task::(prepare) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2001::DEBUG::2012-12-09 12:48:27,723::task::588::TaskManager.Task::(_updateState) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::moving from state preparing -> state finished
>Thread-2001::DEBUG::2012-12-09 12:48:27,723::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2001::DEBUG::2012-12-09 12:48:27,724::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2001::DEBUG::2012-12-09 12:48:27,724::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2001::DEBUG::2012-12-09 12:48:27,724::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2001::DEBUG::2012-12-09 12:48:27,724::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2001::DEBUG::2012-12-09 12:48:27,724::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2001::DEBUG::2012-12-09 12:48:27,725::task::978::TaskManager.Task::(_decref) Task=`089f8920-7964-4d8d-86c5-3ac836a1dc87`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:48:28,751::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:28,790::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163755 s, 62.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:48:29,737::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:48:29,738::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:48:29,836::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:48:29,837::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:48:29,837::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:48:29,837::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:48:30,731::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:30,741::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000596667 s, 6.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:30,797::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:30,838::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170231 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:32,845::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:32,887::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172736 s, 59.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:34,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:34,937::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193488 s, 52.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:36,944::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:36,987::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177671 s, 57.6 MB/s\n'; <rc> = 0
>Thread-2006::DEBUG::2012-12-09 12:48:38,501::task::588::TaskManager.Task::(_updateState) Task=`005b2fe0-802a-4463-a437-31e82f7b5c63`::moving from state init -> state preparing
>Thread-2006::INFO::2012-12-09 12:48:38,501::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2006::INFO::2012-12-09 12:48:38,502::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113778114319', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-2006::DEBUG::2012-12-09 12:48:38,502::task::1172::TaskManager.Task::(prepare) Task=`005b2fe0-802a-4463-a437-31e82f7b5c63`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113778114319', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-2006::DEBUG::2012-12-09 12:48:38,502::task::588::TaskManager.Task::(_updateState) Task=`005b2fe0-802a-4463-a437-31e82f7b5c63`::moving from state preparing -> state finished
>Thread-2006::DEBUG::2012-12-09 12:48:38,502::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2006::DEBUG::2012-12-09 12:48:38,502::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2006::DEBUG::2012-12-09 12:48:38,503::task::978::TaskManager.Task::(_decref) Task=`005b2fe0-802a-4463-a437-31e82f7b5c63`::ref 0 aborting False
>Thread-2008::DEBUG::2012-12-09 12:48:38,651::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2008::DEBUG::2012-12-09 12:48:38,652::task::588::TaskManager.Task::(_updateState) Task=`c52895b0-7bf7-4352-b8e6-d18778397989`::moving from state init -> state preparing
>Thread-2008::INFO::2012-12-09 12:48:38,653::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2008::INFO::2012-12-09 12:48:38,653::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2008::DEBUG::2012-12-09 12:48:38,654::task::1172::TaskManager.Task::(prepare) Task=`c52895b0-7bf7-4352-b8e6-d18778397989`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2008::DEBUG::2012-12-09 12:48:38,654::task::588::TaskManager.Task::(_updateState) Task=`c52895b0-7bf7-4352-b8e6-d18778397989`::moving from state preparing -> state finished
>Thread-2008::DEBUG::2012-12-09 12:48:38,655::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2008::DEBUG::2012-12-09 12:48:38,655::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2008::DEBUG::2012-12-09 12:48:38,655::task::978::TaskManager.Task::(_decref) Task=`c52895b0-7bf7-4352-b8e6-d18778397989`::ref 0 aborting False
>Thread-2009::DEBUG::2012-12-09 12:48:38,672::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2009::DEBUG::2012-12-09 12:48:38,672::task::588::TaskManager.Task::(_updateState) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::moving from state init -> state preparing
>Thread-2009::INFO::2012-12-09 12:48:38,673::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2009::DEBUG::2012-12-09 12:48:38,673::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4d0f41d2-0b8c-414e-b4c4-393e200ab74a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2009::DEBUG::2012-12-09 12:48:38,673::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2009::DEBUG::2012-12-09 12:48:38,673::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2009::DEBUG::2012-12-09 12:48:38,674::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4d0f41d2-0b8c-414e-b4c4-393e200ab74a`::Granted request
>Thread-2009::DEBUG::2012-12-09 12:48:38,674::task::817::TaskManager.Task::(resourceAcquired) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2009::DEBUG::2012-12-09 12:48:38,674::task::978::TaskManager.Task::(_decref) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::ref 1 aborting False
>Thread-2009::DEBUG::2012-12-09 12:48:38,675::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2009::DEBUG::2012-12-09 12:48:38,675::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2009::DEBUG::2012-12-09 12:48:38,784::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2009::WARNING::2012-12-09 12:48:38,786::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2009::DEBUG::2012-12-09 12:48:38,787::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2009::WARNING::2012-12-09 12:48:38,787::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2009::INFO::2012-12-09 12:48:38,788::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2009::DEBUG::2012-12-09 12:48:38,788::task::1172::TaskManager.Task::(prepare) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2009::DEBUG::2012-12-09 12:48:38,788::task::588::TaskManager.Task::(_updateState) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::moving from state preparing -> state finished
>Thread-2009::DEBUG::2012-12-09 12:48:38,788::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2009::DEBUG::2012-12-09 12:48:38,789::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2009::DEBUG::2012-12-09 12:48:38,789::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2009::DEBUG::2012-12-09 12:48:38,789::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2009::DEBUG::2012-12-09 12:48:38,789::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2009::DEBUG::2012-12-09 12:48:38,789::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2009::DEBUG::2012-12-09 12:48:38,790::task::978::TaskManager.Task::(_decref) Task=`95b3aeb6-663e-4975-9794-3a6e0a8096f6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:48:38,994::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:39,042::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023816 s, 43.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:48:39,838::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:48:39,839::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:48:39,950::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:48:39,951::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:48:39,951::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:48:39,952::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:48:40,745::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:40,759::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000834897 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:41,049::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:41,095::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211739 s, 48.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:43,101::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:43,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164279 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:45,150::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:45,189::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157996 s, 64.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:47,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:47,238::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168784 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:49,250::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:49,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180313 s, 56.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:48:49,953::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:48:49,954::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:48:50,064::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:48:50,066::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:48:50,066::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:48:50,067::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2014::DEBUG::2012-12-09 12:48:50,165::task::588::TaskManager.Task::(_updateState) Task=`f8079c04-6833-4b1d-8319-a3a5a0705dc5`::moving from state init -> state preparing
>Thread-2014::INFO::2012-12-09 12:48:50,166::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2014::INFO::2012-12-09 12:48:50,166::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145261287689', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-2014::DEBUG::2012-12-09 12:48:50,166::task::1172::TaskManager.Task::(prepare) Task=`f8079c04-6833-4b1d-8319-a3a5a0705dc5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145261287689', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-2014::DEBUG::2012-12-09 12:48:50,166::task::588::TaskManager.Task::(_updateState) Task=`f8079c04-6833-4b1d-8319-a3a5a0705dc5`::moving from state preparing -> state finished
>Thread-2014::DEBUG::2012-12-09 12:48:50,167::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2014::DEBUG::2012-12-09 12:48:50,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2014::DEBUG::2012-12-09 12:48:50,167::task::978::TaskManager.Task::(_decref) Task=`f8079c04-6833-4b1d-8319-a3a5a0705dc5`::ref 0 aborting False
>Thread-2015::DEBUG::2012-12-09 12:48:50,185::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2015::DEBUG::2012-12-09 12:48:50,186::task::588::TaskManager.Task::(_updateState) Task=`281e93ec-e95d-4044-ad68-ca7f286ff225`::moving from state init -> state preparing
>Thread-2015::INFO::2012-12-09 12:48:50,186::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2015::INFO::2012-12-09 12:48:50,187::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2015::DEBUG::2012-12-09 12:48:50,189::task::1172::TaskManager.Task::(prepare) Task=`281e93ec-e95d-4044-ad68-ca7f286ff225`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2015::DEBUG::2012-12-09 12:48:50,189::task::588::TaskManager.Task::(_updateState) Task=`281e93ec-e95d-4044-ad68-ca7f286ff225`::moving from state preparing -> state finished
>Thread-2015::DEBUG::2012-12-09 12:48:50,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2015::DEBUG::2012-12-09 12:48:50,190::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2015::DEBUG::2012-12-09 12:48:50,190::task::978::TaskManager.Task::(_decref) Task=`281e93ec-e95d-4044-ad68-ca7f286ff225`::ref 0 aborting False
>Thread-2017::DEBUG::2012-12-09 12:48:50,202::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2017::DEBUG::2012-12-09 12:48:50,203::task::588::TaskManager.Task::(_updateState) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::moving from state init -> state preparing
>Thread-2017::INFO::2012-12-09 12:48:50,204::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2017::DEBUG::2012-12-09 12:48:50,204::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9f845cae-0ef2-48dc-8b4d-936acc51fe4d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2017::DEBUG::2012-12-09 12:48:50,205::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2017::DEBUG::2012-12-09 12:48:50,205::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2017::DEBUG::2012-12-09 12:48:50,206::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9f845cae-0ef2-48dc-8b4d-936acc51fe4d`::Granted request
>Thread-2017::DEBUG::2012-12-09 12:48:50,206::task::817::TaskManager.Task::(resourceAcquired) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2017::DEBUG::2012-12-09 12:48:50,207::task::978::TaskManager.Task::(_decref) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::ref 1 aborting False
>Thread-2017::DEBUG::2012-12-09 12:48:50,208::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2017::DEBUG::2012-12-09 12:48:50,208::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2017::DEBUG::2012-12-09 12:48:50,315::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2017::WARNING::2012-12-09 12:48:50,317::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2017::DEBUG::2012-12-09 12:48:50,317::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2017::WARNING::2012-12-09 12:48:50,318::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2017::INFO::2012-12-09 12:48:50,318::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2017::DEBUG::2012-12-09 12:48:50,318::task::1172::TaskManager.Task::(prepare) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2017::DEBUG::2012-12-09 12:48:50,319::task::588::TaskManager.Task::(_updateState) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::moving from state preparing -> state finished
>Thread-2017::DEBUG::2012-12-09 12:48:50,319::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2017::DEBUG::2012-12-09 12:48:50,319::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2017::DEBUG::2012-12-09 12:48:50,319::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2017::DEBUG::2012-12-09 12:48:50,319::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2017::DEBUG::2012-12-09 12:48:50,320::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2017::DEBUG::2012-12-09 12:48:50,320::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2017::DEBUG::2012-12-09 12:48:50,320::task::978::TaskManager.Task::(_decref) Task=`d8e4784c-c779-432f-9175-777993bb2cac`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:48:50,761::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:48:50,769::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000391234 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:51,299::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:51,340::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165263 s, 62.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:53,347::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:53,389::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161365 s, 63.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:55,396::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:55,440::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199726 s, 51.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:57,447::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:57,491::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220138 s, 46.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:48:59,498::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:48:59,540::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176071 s, 58.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:00,068::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:00,069::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:00,164::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:00,165::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:00,165::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:00,166::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:49:00,772::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:00,780::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000458084 s, 8.9 MB/s\n'; <rc> = 0
>Thread-2022::DEBUG::2012-12-09 12:49:01,267::task::588::TaskManager.Task::(_updateState) Task=`6a0ec6e7-6210-4f77-b81b-338835ec8ed1`::moving from state init -> state preparing
>Thread-2022::INFO::2012-12-09 12:49:01,268::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2022::INFO::2012-12-09 12:49:01,268::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00807905197144', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2022::DEBUG::2012-12-09 12:49:01,268::task::1172::TaskManager.Task::(prepare) Task=`6a0ec6e7-6210-4f77-b81b-338835ec8ed1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00807905197144', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2022::DEBUG::2012-12-09 12:49:01,268::task::588::TaskManager.Task::(_updateState) Task=`6a0ec6e7-6210-4f77-b81b-338835ec8ed1`::moving from state preparing -> state finished
>Thread-2022::DEBUG::2012-12-09 12:49:01,268::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2022::DEBUG::2012-12-09 12:49:01,269::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2022::DEBUG::2012-12-09 12:49:01,269::task::978::TaskManager.Task::(_decref) Task=`6a0ec6e7-6210-4f77-b81b-338835ec8ed1`::ref 0 aborting False
>Thread-2024::DEBUG::2012-12-09 12:49:01,442::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2024::DEBUG::2012-12-09 12:49:01,442::task::588::TaskManager.Task::(_updateState) Task=`ac863a1a-abce-41d7-b0b3-6ed7a28233f4`::moving from state init -> state preparing
>Thread-2024::INFO::2012-12-09 12:49:01,443::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2024::INFO::2012-12-09 12:49:01,443::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2024::DEBUG::2012-12-09 12:49:01,443::task::1172::TaskManager.Task::(prepare) Task=`ac863a1a-abce-41d7-b0b3-6ed7a28233f4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2024::DEBUG::2012-12-09 12:49:01,444::task::588::TaskManager.Task::(_updateState) Task=`ac863a1a-abce-41d7-b0b3-6ed7a28233f4`::moving from state preparing -> state finished
>Thread-2024::DEBUG::2012-12-09 12:49:01,444::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2024::DEBUG::2012-12-09 12:49:01,444::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2024::DEBUG::2012-12-09 12:49:01,444::task::978::TaskManager.Task::(_decref) Task=`ac863a1a-abce-41d7-b0b3-6ed7a28233f4`::ref 0 aborting False
>Thread-2025::DEBUG::2012-12-09 12:49:01,460::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2025::DEBUG::2012-12-09 12:49:01,460::task::588::TaskManager.Task::(_updateState) Task=`dee21417-a962-4002-a603-e574ee7c54de`::moving from state init -> state preparing
>Thread-2025::INFO::2012-12-09 12:49:01,460::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2025::DEBUG::2012-12-09 12:49:01,461::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`908696aa-d869-4eed-92d6-a0116633e60d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2025::DEBUG::2012-12-09 12:49:01,461::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2025::DEBUG::2012-12-09 12:49:01,461::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2025::DEBUG::2012-12-09 12:49:01,461::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`908696aa-d869-4eed-92d6-a0116633e60d`::Granted request
>Thread-2025::DEBUG::2012-12-09 12:49:01,462::task::817::TaskManager.Task::(resourceAcquired) Task=`dee21417-a962-4002-a603-e574ee7c54de`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2025::DEBUG::2012-12-09 12:49:01,462::task::978::TaskManager.Task::(_decref) Task=`dee21417-a962-4002-a603-e574ee7c54de`::ref 1 aborting False
>Thread-2025::DEBUG::2012-12-09 12:49:01,463::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2025::DEBUG::2012-12-09 12:49:01,463::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:01,547::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-2025::DEBUG::2012-12-09 12:49:01,570::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2025::WARNING::2012-12-09 12:49:01,573::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2025::DEBUG::2012-12-09 12:49:01,573::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2025::WARNING::2012-12-09 12:49:01,574::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2025::INFO::2012-12-09 12:49:01,575::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2025::DEBUG::2012-12-09 12:49:01,576::task::1172::TaskManager.Task::(prepare) Task=`dee21417-a962-4002-a603-e574ee7c54de`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2025::DEBUG::2012-12-09 12:49:01,576::task::588::TaskManager.Task::(_updateState) Task=`dee21417-a962-4002-a603-e574ee7c54de`::moving from state preparing -> state finished
>Thread-2025::DEBUG::2012-12-09 12:49:01,577::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2025::DEBUG::2012-12-09 12:49:01,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2025::DEBUG::2012-12-09 12:49:01,577::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2025::DEBUG::2012-12-09 12:49:01,578::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2025::DEBUG::2012-12-09 12:49:01,578::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2025::DEBUG::2012-12-09 12:49:01,579::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2025::DEBUG::2012-12-09 12:49:01,579::task::978::TaskManager.Task::(_decref) Task=`dee21417-a962-4002-a603-e574ee7c54de`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:01,605::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353362 s, 29.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:03,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:03,658::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209335 s, 48.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:05,664::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:05,703::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015638 s, 65.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:07,710::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:07,750::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163631 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:09,757::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:09,804::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228083 s, 44.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:10,167::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:10,168::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:10,272::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:10,274::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:10,274::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:10,275::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:49:10,782::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:49:10,783::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:49:10,784::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:10,796::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000722176 s, 2.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:49:10,796::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000722176 s, 2.8 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:49:10,797::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:49:10,797::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:49:10,797::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:49:10,797::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,797::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,798::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,798::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,798::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,798::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:10,897::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:49:10,898::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:49:10,899::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:49:10,899::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:49:10,899::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:49:10,900::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:49:10,900::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:10,999::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:49:10,999::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:11,011::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000758411 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:11,811::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:11,862::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265485 s, 38.6 MB/s\n'; <rc> = 0
>Thread-2030::DEBUG::2012-12-09 12:49:11,938::task::588::TaskManager.Task::(_updateState) Task=`afd22189-87b0-4310-81cc-3990b9b804a6`::moving from state init -> state preparing
>Thread-2030::INFO::2012-12-09 12:49:11,939::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2030::INFO::2012-12-09 12:49:11,940::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121598243713', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2030::DEBUG::2012-12-09 12:49:11,940::task::1172::TaskManager.Task::(prepare) Task=`afd22189-87b0-4310-81cc-3990b9b804a6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121598243713', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2030::DEBUG::2012-12-09 12:49:11,941::task::588::TaskManager.Task::(_updateState) Task=`afd22189-87b0-4310-81cc-3990b9b804a6`::moving from state preparing -> state finished
>Thread-2030::DEBUG::2012-12-09 12:49:11,941::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2030::DEBUG::2012-12-09 12:49:11,941::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2030::DEBUG::2012-12-09 12:49:11,942::task::978::TaskManager.Task::(_decref) Task=`afd22189-87b0-4310-81cc-3990b9b804a6`::ref 0 aborting False
>Thread-2032::DEBUG::2012-12-09 12:49:12,020::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2032::DEBUG::2012-12-09 12:49:12,020::task::588::TaskManager.Task::(_updateState) Task=`b0396c37-a027-4d86-8f78-0ea40ce03933`::moving from state init -> state preparing
>Thread-2032::INFO::2012-12-09 12:49:12,020::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2032::INFO::2012-12-09 12:49:12,021::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2032::DEBUG::2012-12-09 12:49:12,021::task::1172::TaskManager.Task::(prepare) Task=`b0396c37-a027-4d86-8f78-0ea40ce03933`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2032::DEBUG::2012-12-09 12:49:12,021::task::588::TaskManager.Task::(_updateState) Task=`b0396c37-a027-4d86-8f78-0ea40ce03933`::moving from state preparing -> state finished
>Thread-2032::DEBUG::2012-12-09 12:49:12,021::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2032::DEBUG::2012-12-09 12:49:12,021::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2032::DEBUG::2012-12-09 12:49:12,021::task::978::TaskManager.Task::(_decref) Task=`b0396c37-a027-4d86-8f78-0ea40ce03933`::ref 0 aborting False
>Thread-2033::DEBUG::2012-12-09 12:49:12,041::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2033::DEBUG::2012-12-09 12:49:12,042::task::588::TaskManager.Task::(_updateState) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::moving from state init -> state preparing
>Thread-2033::INFO::2012-12-09 12:49:12,042::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2033::DEBUG::2012-12-09 12:49:12,043::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd2be65e-3472-4f0a-97dc-10a36ca5fba5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2033::DEBUG::2012-12-09 12:49:12,044::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2033::DEBUG::2012-12-09 12:49:12,044::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2033::DEBUG::2012-12-09 12:49:12,045::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd2be65e-3472-4f0a-97dc-10a36ca5fba5`::Granted request
>Thread-2033::DEBUG::2012-12-09 12:49:12,045::task::817::TaskManager.Task::(resourceAcquired) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2033::DEBUG::2012-12-09 12:49:12,046::task::978::TaskManager.Task::(_decref) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::ref 1 aborting False
>Thread-2033::DEBUG::2012-12-09 12:49:12,047::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2033::DEBUG::2012-12-09 12:49:12,048::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2033::DEBUG::2012-12-09 12:49:12,144::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2033::WARNING::2012-12-09 12:49:12,146::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2033::DEBUG::2012-12-09 12:49:12,146::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2033::WARNING::2012-12-09 12:49:12,146::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2033::INFO::2012-12-09 12:49:12,147::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2033::DEBUG::2012-12-09 12:49:12,147::task::1172::TaskManager.Task::(prepare) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2033::DEBUG::2012-12-09 12:49:12,147::task::588::TaskManager.Task::(_updateState) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::moving from state preparing -> state finished
>Thread-2033::DEBUG::2012-12-09 12:49:12,148::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2033::DEBUG::2012-12-09 12:49:12,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2033::DEBUG::2012-12-09 12:49:12,148::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2033::DEBUG::2012-12-09 12:49:12,148::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2033::DEBUG::2012-12-09 12:49:12,148::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2033::DEBUG::2012-12-09 12:49:12,149::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2033::DEBUG::2012-12-09 12:49:12,149::task::978::TaskManager.Task::(_decref) Task=`4641651d-5e9b-4861-8d06-011b80bab4a1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:13,868::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:13,920::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211801 s, 48.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:15,927::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:15,970::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017131 s, 59.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:17,976::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:18,021::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180403 s, 56.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:20,028::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:20,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159742 s, 64.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:20,276::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:20,277::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:20,385::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:20,387::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:20,388::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:20,388::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:49:21,014::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:21,027::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000749329 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:22,073::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:22,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216619 s, 47.3 MB/s\n'; <rc> = 0
>Thread-2038::DEBUG::2012-12-09 12:49:23,354::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2038::DEBUG::2012-12-09 12:49:23,355::task::588::TaskManager.Task::(_updateState) Task=`feea48b9-656b-453b-92c5-3d5687269a3a`::moving from state init -> state preparing
>Thread-2038::INFO::2012-12-09 12:49:23,355::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2038::INFO::2012-12-09 12:49:23,355::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2038::DEBUG::2012-12-09 12:49:23,355::task::1172::TaskManager.Task::(prepare) Task=`feea48b9-656b-453b-92c5-3d5687269a3a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2038::DEBUG::2012-12-09 12:49:23,356::task::588::TaskManager.Task::(_updateState) Task=`feea48b9-656b-453b-92c5-3d5687269a3a`::moving from state preparing -> state finished
>Thread-2038::DEBUG::2012-12-09 12:49:23,356::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2038::DEBUG::2012-12-09 12:49:23,356::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2038::DEBUG::2012-12-09 12:49:23,356::task::978::TaskManager.Task::(_decref) Task=`feea48b9-656b-453b-92c5-3d5687269a3a`::ref 0 aborting False
>Thread-2039::DEBUG::2012-12-09 12:49:23,368::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2039::DEBUG::2012-12-09 12:49:23,369::task::588::TaskManager.Task::(_updateState) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::moving from state init -> state preparing
>Thread-2039::INFO::2012-12-09 12:49:23,369::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2039::DEBUG::2012-12-09 12:49:23,369::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ee0b877-9903-45aa-add5-c8c42a8d5719`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2039::DEBUG::2012-12-09 12:49:23,369::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2039::DEBUG::2012-12-09 12:49:23,370::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2039::DEBUG::2012-12-09 12:49:23,370::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4ee0b877-9903-45aa-add5-c8c42a8d5719`::Granted request
>Thread-2039::DEBUG::2012-12-09 12:49:23,370::task::817::TaskManager.Task::(resourceAcquired) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2039::DEBUG::2012-12-09 12:49:23,370::task::978::TaskManager.Task::(_decref) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::ref 1 aborting False
>Thread-2039::DEBUG::2012-12-09 12:49:23,371::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2039::DEBUG::2012-12-09 12:49:23,372::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2040::DEBUG::2012-12-09 12:49:23,465::task::588::TaskManager.Task::(_updateState) Task=`97f24346-6fd5-4326-a058-bce67b901a5d`::moving from state init -> state preparing
>Thread-2040::INFO::2012-12-09 12:49:23,465::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2040::INFO::2012-12-09 12:49:23,466::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134589672089', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2040::DEBUG::2012-12-09 12:49:23,466::task::1172::TaskManager.Task::(prepare) Task=`97f24346-6fd5-4326-a058-bce67b901a5d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134589672089', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2040::DEBUG::2012-12-09 12:49:23,466::task::588::TaskManager.Task::(_updateState) Task=`97f24346-6fd5-4326-a058-bce67b901a5d`::moving from state preparing -> state finished
>Thread-2040::DEBUG::2012-12-09 12:49:23,467::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2040::DEBUG::2012-12-09 12:49:23,467::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2040::DEBUG::2012-12-09 12:49:23,467::task::978::TaskManager.Task::(_decref) Task=`97f24346-6fd5-4326-a058-bce67b901a5d`::ref 0 aborting False
>Thread-2039::DEBUG::2012-12-09 12:49:23,471::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2039::WARNING::2012-12-09 12:49:23,473::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2039::DEBUG::2012-12-09 12:49:23,473::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2039::WARNING::2012-12-09 12:49:23,474::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2039::INFO::2012-12-09 12:49:23,474::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2039::DEBUG::2012-12-09 12:49:23,474::task::1172::TaskManager.Task::(prepare) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2039::DEBUG::2012-12-09 12:49:23,474::task::588::TaskManager.Task::(_updateState) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::moving from state preparing -> state finished
>Thread-2039::DEBUG::2012-12-09 12:49:23,475::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2039::DEBUG::2012-12-09 12:49:23,475::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2039::DEBUG::2012-12-09 12:49:23,475::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2039::DEBUG::2012-12-09 12:49:23,476::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2039::DEBUG::2012-12-09 12:49:23,476::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2039::DEBUG::2012-12-09 12:49:23,477::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2039::DEBUG::2012-12-09 12:49:23,477::task::978::TaskManager.Task::(_decref) Task=`25f47c12-885d-4513-9325-2ea61a775fb6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:24,127::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:24,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159315 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:26,173::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:26,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213152 s, 48.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:28,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:28,265::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162484 s, 63.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:30,272::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:30,322::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01686 s, 60.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:30,389::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:49:30,389::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:30,390::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:30,488::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:30,489::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:30,489::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:30,490::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:49:31,030::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:31,046::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000666951 s, 6.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:32,330::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:32,382::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023829 s, 43.0 MB/s\n'; <rc> = 0
>Thread-2046::DEBUG::2012-12-09 12:49:34,343::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2046::DEBUG::2012-12-09 12:49:34,344::task::588::TaskManager.Task::(_updateState) Task=`398d27f5-934a-4013-b34b-d39ccb26921d`::moving from state init -> state preparing
>Thread-2046::INFO::2012-12-09 12:49:34,344::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2046::INFO::2012-12-09 12:49:34,345::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2046::DEBUG::2012-12-09 12:49:34,345::task::1172::TaskManager.Task::(prepare) Task=`398d27f5-934a-4013-b34b-d39ccb26921d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2046::DEBUG::2012-12-09 12:49:34,346::task::588::TaskManager.Task::(_updateState) Task=`398d27f5-934a-4013-b34b-d39ccb26921d`::moving from state preparing -> state finished
>Thread-2046::DEBUG::2012-12-09 12:49:34,346::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2046::DEBUG::2012-12-09 12:49:34,347::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2046::DEBUG::2012-12-09 12:49:34,347::task::978::TaskManager.Task::(_decref) Task=`398d27f5-934a-4013-b34b-d39ccb26921d`::ref 0 aborting False
>Thread-2047::DEBUG::2012-12-09 12:49:34,359::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2047::DEBUG::2012-12-09 12:49:34,360::task::588::TaskManager.Task::(_updateState) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::moving from state init -> state preparing
>Thread-2047::INFO::2012-12-09 12:49:34,360::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2047::DEBUG::2012-12-09 12:49:34,361::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a9ae780e-9647-4d48-a415-0c79e5ac437a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2047::DEBUG::2012-12-09 12:49:34,361::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2047::DEBUG::2012-12-09 12:49:34,361::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2047::DEBUG::2012-12-09 12:49:34,361::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a9ae780e-9647-4d48-a415-0c79e5ac437a`::Granted request
>Thread-2047::DEBUG::2012-12-09 12:49:34,361::task::817::TaskManager.Task::(resourceAcquired) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2047::DEBUG::2012-12-09 12:49:34,362::task::978::TaskManager.Task::(_decref) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::ref 1 aborting False
>Thread-2047::DEBUG::2012-12-09 12:49:34,362::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2047::DEBUG::2012-12-09 12:49:34,363::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:34,388::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-2048::DEBUG::2012-12-09 12:49:34,395::task::588::TaskManager.Task::(_updateState) Task=`034ea747-b3c3-47f5-9406-39531a5f8bd5`::moving from state init -> state preparing
>Thread-2048::INFO::2012-12-09 12:49:34,397::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2048::INFO::2012-12-09 12:49:34,397::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160632133484', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-2048::DEBUG::2012-12-09 12:49:34,398::task::1172::TaskManager.Task::(prepare) Task=`034ea747-b3c3-47f5-9406-39531a5f8bd5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160632133484', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-2048::DEBUG::2012-12-09 12:49:34,398::task::588::TaskManager.Task::(_updateState) Task=`034ea747-b3c3-47f5-9406-39531a5f8bd5`::moving from state preparing -> state finished
>Thread-2048::DEBUG::2012-12-09 12:49:34,398::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2048::DEBUG::2012-12-09 12:49:34,398::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2048::DEBUG::2012-12-09 12:49:34,398::task::978::TaskManager.Task::(_decref) Task=`034ea747-b3c3-47f5-9406-39531a5f8bd5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:34,442::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250797 s, 40.8 MB/s\n'; <rc> = 0
>Thread-2047::DEBUG::2012-12-09 12:49:34,477::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2047::WARNING::2012-12-09 12:49:34,478::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2047::DEBUG::2012-12-09 12:49:34,478::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2047::WARNING::2012-12-09 12:49:34,479::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2047::INFO::2012-12-09 12:49:34,479::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2047::DEBUG::2012-12-09 12:49:34,479::task::1172::TaskManager.Task::(prepare) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2047::DEBUG::2012-12-09 12:49:34,480::task::588::TaskManager.Task::(_updateState) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::moving from state preparing -> state finished
>Thread-2047::DEBUG::2012-12-09 12:49:34,480::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2047::DEBUG::2012-12-09 12:49:34,480::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2047::DEBUG::2012-12-09 12:49:34,480::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2047::DEBUG::2012-12-09 12:49:34,481::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2047::DEBUG::2012-12-09 12:49:34,481::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2047::DEBUG::2012-12-09 12:49:34,481::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2047::DEBUG::2012-12-09 12:49:34,481::task::978::TaskManager.Task::(_decref) Task=`ad737d26-796c-4fdf-ad06-e6ebdbb23247`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:36,449::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:36,493::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185823 s, 55.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:38,500::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:38,550::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225556 s, 45.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:40,491::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:40,492::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:40,556::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:40,598::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:40,599::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:40,599::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:40,600::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:49:40,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192012 s, 53.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:49:41,049::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:41,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000507591 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:42,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:42,652::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168989 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:44,659::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:44,701::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163035 s, 62.8 MB/s\n'; <rc> = 0
>Thread-2054::DEBUG::2012-12-09 12:49:46,470::task::588::TaskManager.Task::(_updateState) Task=`58a065b1-6dfd-461c-8f6a-b72fa00035c7`::moving from state init -> state preparing
>Thread-2054::INFO::2012-12-09 12:49:46,470::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2054::INFO::2012-12-09 12:49:46,470::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119359493256', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-2054::DEBUG::2012-12-09 12:49:46,471::task::1172::TaskManager.Task::(prepare) Task=`58a065b1-6dfd-461c-8f6a-b72fa00035c7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119359493256', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-2054::DEBUG::2012-12-09 12:49:46,471::task::588::TaskManager.Task::(_updateState) Task=`58a065b1-6dfd-461c-8f6a-b72fa00035c7`::moving from state preparing -> state finished
>Thread-2054::DEBUG::2012-12-09 12:49:46,471::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2054::DEBUG::2012-12-09 12:49:46,471::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2054::DEBUG::2012-12-09 12:49:46,471::task::978::TaskManager.Task::(_decref) Task=`58a065b1-6dfd-461c-8f6a-b72fa00035c7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:46,709::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:46,748::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164171 s, 62.4 MB/s\n'; <rc> = 0
>Thread-2056::DEBUG::2012-12-09 12:49:46,961::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2056::DEBUG::2012-12-09 12:49:46,962::task::588::TaskManager.Task::(_updateState) Task=`e3d1bca2-475e-4591-a23c-a3e7d7e2345a`::moving from state init -> state preparing
>Thread-2056::INFO::2012-12-09 12:49:46,962::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2056::INFO::2012-12-09 12:49:46,962::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2056::DEBUG::2012-12-09 12:49:46,962::task::1172::TaskManager.Task::(prepare) Task=`e3d1bca2-475e-4591-a23c-a3e7d7e2345a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2056::DEBUG::2012-12-09 12:49:46,962::task::588::TaskManager.Task::(_updateState) Task=`e3d1bca2-475e-4591-a23c-a3e7d7e2345a`::moving from state preparing -> state finished
>Thread-2056::DEBUG::2012-12-09 12:49:46,963::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2056::DEBUG::2012-12-09 12:49:46,963::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2056::DEBUG::2012-12-09 12:49:46,963::task::978::TaskManager.Task::(_decref) Task=`e3d1bca2-475e-4591-a23c-a3e7d7e2345a`::ref 0 aborting False
>Thread-2057::DEBUG::2012-12-09 12:49:46,975::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2057::DEBUG::2012-12-09 12:49:46,975::task::588::TaskManager.Task::(_updateState) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::moving from state init -> state preparing
>Thread-2057::INFO::2012-12-09 12:49:46,975::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2057::DEBUG::2012-12-09 12:49:46,976::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d3bec445-f2a6-43a8-a206-a25a5a113159`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2057::DEBUG::2012-12-09 12:49:46,976::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2057::DEBUG::2012-12-09 12:49:46,976::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2057::DEBUG::2012-12-09 12:49:46,976::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d3bec445-f2a6-43a8-a206-a25a5a113159`::Granted request
>Thread-2057::DEBUG::2012-12-09 12:49:46,977::task::817::TaskManager.Task::(resourceAcquired) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2057::DEBUG::2012-12-09 12:49:46,977::task::978::TaskManager.Task::(_decref) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::ref 1 aborting False
>Thread-2057::DEBUG::2012-12-09 12:49:46,978::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2057::DEBUG::2012-12-09 12:49:46,978::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2057::DEBUG::2012-12-09 12:49:47,074::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2057::WARNING::2012-12-09 12:49:47,075::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2057::DEBUG::2012-12-09 12:49:47,076::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2057::WARNING::2012-12-09 12:49:47,076::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2057::INFO::2012-12-09 12:49:47,077::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2057::DEBUG::2012-12-09 12:49:47,077::task::1172::TaskManager.Task::(prepare) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2057::DEBUG::2012-12-09 12:49:47,077::task::588::TaskManager.Task::(_updateState) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::moving from state preparing -> state finished
>Thread-2057::DEBUG::2012-12-09 12:49:47,077::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2057::DEBUG::2012-12-09 12:49:47,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2057::DEBUG::2012-12-09 12:49:47,078::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2057::DEBUG::2012-12-09 12:49:47,078::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2057::DEBUG::2012-12-09 12:49:47,078::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2057::DEBUG::2012-12-09 12:49:47,079::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2057::DEBUG::2012-12-09 12:49:47,079::task::978::TaskManager.Task::(_decref) Task=`cd088375-7331-43d3-ba59-2b3d624c8b37`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:48,755::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:48,799::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187232 s, 54.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:49:50,601::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:49:50,601::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:49:50,702::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:49:50,703::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:49:50,703::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:49:50,704::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:49:50,807::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:50,846::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168496 s, 60.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:49:51,064::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:49:51,075::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000354165 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:52,853::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:52,894::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170316 s, 60.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:54,901::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:54,941::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166619 s, 61.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:49:56,947::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:57,004::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228036 s, 44.9 MB/s\n'; <rc> = 0
>Thread-2062::DEBUG::2012-12-09 12:49:57,314::task::588::TaskManager.Task::(_updateState) Task=`ef4b2f31-85ea-4887-a446-f113516ce1b8`::moving from state init -> state preparing
>Thread-2062::INFO::2012-12-09 12:49:57,314::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2062::INFO::2012-12-09 12:49:57,315::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116939544678', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2062::DEBUG::2012-12-09 12:49:57,315::task::1172::TaskManager.Task::(prepare) Task=`ef4b2f31-85ea-4887-a446-f113516ce1b8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116939544678', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2062::DEBUG::2012-12-09 12:49:57,315::task::588::TaskManager.Task::(_updateState) Task=`ef4b2f31-85ea-4887-a446-f113516ce1b8`::moving from state preparing -> state finished
>Thread-2062::DEBUG::2012-12-09 12:49:57,315::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2062::DEBUG::2012-12-09 12:49:57,316::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2062::DEBUG::2012-12-09 12:49:57,316::task::978::TaskManager.Task::(_decref) Task=`ef4b2f31-85ea-4887-a446-f113516ce1b8`::ref 0 aborting False
>Thread-2064::DEBUG::2012-12-09 12:49:58,484::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2064::DEBUG::2012-12-09 12:49:58,484::task::588::TaskManager.Task::(_updateState) Task=`3e477b04-3ecc-421e-a274-4f67c3b76892`::moving from state init -> state preparing
>Thread-2064::INFO::2012-12-09 12:49:58,485::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2064::INFO::2012-12-09 12:49:58,485::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2064::DEBUG::2012-12-09 12:49:58,485::task::1172::TaskManager.Task::(prepare) Task=`3e477b04-3ecc-421e-a274-4f67c3b76892`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2064::DEBUG::2012-12-09 12:49:58,485::task::588::TaskManager.Task::(_updateState) Task=`3e477b04-3ecc-421e-a274-4f67c3b76892`::moving from state preparing -> state finished
>Thread-2064::DEBUG::2012-12-09 12:49:58,486::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2064::DEBUG::2012-12-09 12:49:58,486::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2064::DEBUG::2012-12-09 12:49:58,486::task::978::TaskManager.Task::(_decref) Task=`3e477b04-3ecc-421e-a274-4f67c3b76892`::ref 0 aborting False
>Thread-2065::DEBUG::2012-12-09 12:49:58,561::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2065::DEBUG::2012-12-09 12:49:58,568::task::588::TaskManager.Task::(_updateState) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::moving from state init -> state preparing
>Thread-2065::INFO::2012-12-09 12:49:58,568::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2065::DEBUG::2012-12-09 12:49:58,569::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c12fbb4f-4bfd-4bfd-88ad-11ebf82479f4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2065::DEBUG::2012-12-09 12:49:58,569::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2065::DEBUG::2012-12-09 12:49:58,569::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2065::DEBUG::2012-12-09 12:49:58,569::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c12fbb4f-4bfd-4bfd-88ad-11ebf82479f4`::Granted request
>Thread-2065::DEBUG::2012-12-09 12:49:58,570::task::817::TaskManager.Task::(resourceAcquired) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2065::DEBUG::2012-12-09 12:49:58,570::task::978::TaskManager.Task::(_decref) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::ref 1 aborting False
>Thread-2065::DEBUG::2012-12-09 12:49:58,571::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2065::DEBUG::2012-12-09 12:49:58,571::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2065::DEBUG::2012-12-09 12:49:58,759::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2065::WARNING::2012-12-09 12:49:58,760::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2065::DEBUG::2012-12-09 12:49:58,760::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2065::WARNING::2012-12-09 12:49:58,761::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2065::INFO::2012-12-09 12:49:58,761::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2065::DEBUG::2012-12-09 12:49:58,762::task::1172::TaskManager.Task::(prepare) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2065::DEBUG::2012-12-09 12:49:58,762::task::588::TaskManager.Task::(_updateState) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::moving from state preparing -> state finished
>Thread-2065::DEBUG::2012-12-09 12:49:58,762::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2065::DEBUG::2012-12-09 12:49:58,762::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2065::DEBUG::2012-12-09 12:49:58,763::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2065::DEBUG::2012-12-09 12:49:58,763::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2065::DEBUG::2012-12-09 12:49:58,763::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2065::DEBUG::2012-12-09 12:49:58,764::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2065::DEBUG::2012-12-09 12:49:58,764::task::978::TaskManager.Task::(_decref) Task=`a999e7ff-6d95-4f73-a691-acad4ae0d0d8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:49:59,013::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:49:59,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239628 s, 42.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:50:00,704::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:00,705::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:50:00,843::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:50:00,844::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:00,844::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:00,845::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:50:01,078::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:01,103::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:01,109::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00175422 s, 2.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:01,176::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0301002 s, 34.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:03,184::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:03,252::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268775 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:05,261::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:05,301::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169285 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:07,308::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:07,351::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187885 s, 54.5 MB/s\n'; <rc> = 0
>Thread-2070::DEBUG::2012-12-09 12:50:08,154::task::588::TaskManager.Task::(_updateState) Task=`192498ce-99e1-44f3-a405-df9369a47d2d`::moving from state init -> state preparing
>Thread-2070::INFO::2012-12-09 12:50:08,154::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2070::INFO::2012-12-09 12:50:08,155::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0319261550903', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-2070::DEBUG::2012-12-09 12:50:08,155::task::1172::TaskManager.Task::(prepare) Task=`192498ce-99e1-44f3-a405-df9369a47d2d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0319261550903', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-2070::DEBUG::2012-12-09 12:50:08,156::task::588::TaskManager.Task::(_updateState) Task=`192498ce-99e1-44f3-a405-df9369a47d2d`::moving from state preparing -> state finished
>Thread-2070::DEBUG::2012-12-09 12:50:08,156::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2070::DEBUG::2012-12-09 12:50:08,156::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2070::DEBUG::2012-12-09 12:50:08,156::task::978::TaskManager.Task::(_decref) Task=`192498ce-99e1-44f3-a405-df9369a47d2d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:09,357::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:09,400::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158801 s, 64.5 MB/s\n'; <rc> = 0
>Thread-2072::DEBUG::2012-12-09 12:50:09,462::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2072::DEBUG::2012-12-09 12:50:09,463::task::588::TaskManager.Task::(_updateState) Task=`f8ceba97-834e-4aaf-8003-dfed29a0f8c8`::moving from state init -> state preparing
>Thread-2072::INFO::2012-12-09 12:50:09,464::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2072::INFO::2012-12-09 12:50:09,464::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2072::DEBUG::2012-12-09 12:50:09,465::task::1172::TaskManager.Task::(prepare) Task=`f8ceba97-834e-4aaf-8003-dfed29a0f8c8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2072::DEBUG::2012-12-09 12:50:09,465::task::588::TaskManager.Task::(_updateState) Task=`f8ceba97-834e-4aaf-8003-dfed29a0f8c8`::moving from state preparing -> state finished
>Thread-2072::DEBUG::2012-12-09 12:50:09,466::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2072::DEBUG::2012-12-09 12:50:09,466::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2072::DEBUG::2012-12-09 12:50:09,467::task::978::TaskManager.Task::(_decref) Task=`f8ceba97-834e-4aaf-8003-dfed29a0f8c8`::ref 0 aborting False
>Thread-2073::DEBUG::2012-12-09 12:50:09,479::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2073::DEBUG::2012-12-09 12:50:09,479::task::588::TaskManager.Task::(_updateState) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::moving from state init -> state preparing
>Thread-2073::INFO::2012-12-09 12:50:09,479::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2073::DEBUG::2012-12-09 12:50:09,480::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9fefa597-62e5-440a-85ae-8db54ffe390c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2073::DEBUG::2012-12-09 12:50:09,480::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2073::DEBUG::2012-12-09 12:50:09,480::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2073::DEBUG::2012-12-09 12:50:09,480::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9fefa597-62e5-440a-85ae-8db54ffe390c`::Granted request
>Thread-2073::DEBUG::2012-12-09 12:50:09,481::task::817::TaskManager.Task::(resourceAcquired) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2073::DEBUG::2012-12-09 12:50:09,481::task::978::TaskManager.Task::(_decref) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::ref 1 aborting False
>Thread-2073::DEBUG::2012-12-09 12:50:09,482::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2073::DEBUG::2012-12-09 12:50:09,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2073::DEBUG::2012-12-09 12:50:09,593::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2073::WARNING::2012-12-09 12:50:09,595::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2073::DEBUG::2012-12-09 12:50:09,596::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2073::WARNING::2012-12-09 12:50:09,597::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2073::INFO::2012-12-09 12:50:09,597::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2073::DEBUG::2012-12-09 12:50:09,598::task::1172::TaskManager.Task::(prepare) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2073::DEBUG::2012-12-09 12:50:09,598::task::588::TaskManager.Task::(_updateState) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::moving from state preparing -> state finished
>Thread-2073::DEBUG::2012-12-09 12:50:09,598::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2073::DEBUG::2012-12-09 12:50:09,598::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2073::DEBUG::2012-12-09 12:50:09,599::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2073::DEBUG::2012-12-09 12:50:09,599::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2073::DEBUG::2012-12-09 12:50:09,599::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2073::DEBUG::2012-12-09 12:50:09,599::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2073::DEBUG::2012-12-09 12:50:09,599::task::978::TaskManager.Task::(_decref) Task=`547fc7a0-188d-495d-8591-aaa2dca6b369`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:50:10,846::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:10,846::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:50:10,945::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:50:10,947::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:10,948::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:10,949::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:50:11,113::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:11,122::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000406693 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:11,407::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:11,455::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225084 s, 45.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:13,462::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:13,502::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171853 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:15,509::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:15,548::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163344 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:17,554::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:17,594::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166475 s, 61.5 MB/s\n'; <rc> = 0
>Thread-2078::DEBUG::2012-12-09 12:50:19,076::task::588::TaskManager.Task::(_updateState) Task=`23451d4e-cd6c-404d-a7ee-99a012febab4`::moving from state init -> state preparing
>Thread-2078::INFO::2012-12-09 12:50:19,077::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2078::INFO::2012-12-09 12:50:19,077::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00971293449402', 'lastCheck': '8.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-2078::DEBUG::2012-12-09 12:50:19,077::task::1172::TaskManager.Task::(prepare) Task=`23451d4e-cd6c-404d-a7ee-99a012febab4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00971293449402', 'lastCheck': '8.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.1', 'code': 358, 'valid': False}}
>Thread-2078::DEBUG::2012-12-09 12:50:19,078::task::588::TaskManager.Task::(_updateState) Task=`23451d4e-cd6c-404d-a7ee-99a012febab4`::moving from state preparing -> state finished
>Thread-2078::DEBUG::2012-12-09 12:50:19,078::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2078::DEBUG::2012-12-09 12:50:19,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2078::DEBUG::2012-12-09 12:50:19,078::task::978::TaskManager.Task::(_decref) Task=`23451d4e-cd6c-404d-a7ee-99a012febab4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:19,601::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:19,643::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175026 s, 58.5 MB/s\n'; <rc> = 0
>Thread-2080::DEBUG::2012-12-09 12:50:20,718::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2080::DEBUG::2012-12-09 12:50:20,719::task::588::TaskManager.Task::(_updateState) Task=`e3891cd6-444a-46fd-89b5-30a5a9160a46`::moving from state init -> state preparing
>Thread-2080::INFO::2012-12-09 12:50:20,719::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2080::INFO::2012-12-09 12:50:20,720::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2080::DEBUG::2012-12-09 12:50:20,720::task::1172::TaskManager.Task::(prepare) Task=`e3891cd6-444a-46fd-89b5-30a5a9160a46`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2080::DEBUG::2012-12-09 12:50:20,720::task::588::TaskManager.Task::(_updateState) Task=`e3891cd6-444a-46fd-89b5-30a5a9160a46`::moving from state preparing -> state finished
>Thread-2080::DEBUG::2012-12-09 12:50:20,720::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2080::DEBUG::2012-12-09 12:50:20,721::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2080::DEBUG::2012-12-09 12:50:20,721::task::978::TaskManager.Task::(_decref) Task=`e3891cd6-444a-46fd-89b5-30a5a9160a46`::ref 0 aborting False
>Thread-2081::DEBUG::2012-12-09 12:50:20,733::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2081::DEBUG::2012-12-09 12:50:20,733::task::588::TaskManager.Task::(_updateState) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::moving from state init -> state preparing
>Thread-2081::INFO::2012-12-09 12:50:20,733::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2081::DEBUG::2012-12-09 12:50:20,734::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`327d528f-8127-48b5-b59e-463ad4fe07eb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2081::DEBUG::2012-12-09 12:50:20,734::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2081::DEBUG::2012-12-09 12:50:20,734::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2081::DEBUG::2012-12-09 12:50:20,734::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`327d528f-8127-48b5-b59e-463ad4fe07eb`::Granted request
>Thread-2081::DEBUG::2012-12-09 12:50:20,735::task::817::TaskManager.Task::(resourceAcquired) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2081::DEBUG::2012-12-09 12:50:20,735::task::978::TaskManager.Task::(_decref) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::ref 1 aborting False
>Thread-2081::DEBUG::2012-12-09 12:50:20,735::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2081::DEBUG::2012-12-09 12:50:20,736::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2081::DEBUG::2012-12-09 12:50:20,835::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2081::WARNING::2012-12-09 12:50:20,836::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2081::DEBUG::2012-12-09 12:50:20,836::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2081::WARNING::2012-12-09 12:50:20,837::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2081::INFO::2012-12-09 12:50:20,837::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2081::DEBUG::2012-12-09 12:50:20,837::task::1172::TaskManager.Task::(prepare) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2081::DEBUG::2012-12-09 12:50:20,837::task::588::TaskManager.Task::(_updateState) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::moving from state preparing -> state finished
>Thread-2081::DEBUG::2012-12-09 12:50:20,838::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2081::DEBUG::2012-12-09 12:50:20,838::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2081::DEBUG::2012-12-09 12:50:20,838::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2081::DEBUG::2012-12-09 12:50:20,838::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2081::DEBUG::2012-12-09 12:50:20,839::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2081::DEBUG::2012-12-09 12:50:20,839::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2081::DEBUG::2012-12-09 12:50:20,839::task::978::TaskManager.Task::(_decref) Task=`7e2c3f77-008d-447b-b665-9130890fa811`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:50:20,950::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:20,951::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:50:21,045::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:50:21,046::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:21,047::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:21,047::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:50:21,126::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:21,139::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000639486 s, 6.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:21,649::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:21,691::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168256 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:23,698::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:23,737::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166265 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:25,744::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:25,793::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211535 s, 48.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:27,799::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:27,840::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171093 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:29,847::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:29,886::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171217 s, 59.8 MB/s\n'; <rc> = 0
>Thread-2086::DEBUG::2012-12-09 12:50:30,371::task::588::TaskManager.Task::(_updateState) Task=`f9410902-d7bc-4f84-8961-c0380c649c0f`::moving from state init -> state preparing
>Thread-2086::INFO::2012-12-09 12:50:30,371::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2086::INFO::2012-12-09 12:50:30,371::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130281448364', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2086::DEBUG::2012-12-09 12:50:30,372::task::1172::TaskManager.Task::(prepare) Task=`f9410902-d7bc-4f84-8961-c0380c649c0f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130281448364', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2086::DEBUG::2012-12-09 12:50:30,372::task::588::TaskManager.Task::(_updateState) Task=`f9410902-d7bc-4f84-8961-c0380c649c0f`::moving from state preparing -> state finished
>Thread-2086::DEBUG::2012-12-09 12:50:30,372::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2086::DEBUG::2012-12-09 12:50:30,372::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2086::DEBUG::2012-12-09 12:50:30,372::task::978::TaskManager.Task::(_decref) Task=`f9410902-d7bc-4f84-8961-c0380c649c0f`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:50:31,048::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:31,049::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:50:31,142::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-50::DEBUG::2012-12-09 12:50:31,143::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-51::WARNING::2012-12-09 12:50:31,150::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:31,153::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:31,157::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 12:50:31,159::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000731625 s, 5.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:31,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:31,933::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163438 s, 62.7 MB/s\n'; <rc> = 0
>Thread-2089::DEBUG::2012-12-09 12:50:33,655::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2089::DEBUG::2012-12-09 12:50:33,656::task::588::TaskManager.Task::(_updateState) Task=`e1ec9f0e-87d6-49d1-afbd-e28aef0f99a1`::moving from state init -> state preparing
>Thread-2089::INFO::2012-12-09 12:50:33,656::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2089::INFO::2012-12-09 12:50:33,656::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2089::DEBUG::2012-12-09 12:50:33,657::task::1172::TaskManager.Task::(prepare) Task=`e1ec9f0e-87d6-49d1-afbd-e28aef0f99a1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2089::DEBUG::2012-12-09 12:50:33,657::task::588::TaskManager.Task::(_updateState) Task=`e1ec9f0e-87d6-49d1-afbd-e28aef0f99a1`::moving from state preparing -> state finished
>Thread-2089::DEBUG::2012-12-09 12:50:33,657::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2089::DEBUG::2012-12-09 12:50:33,657::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2089::DEBUG::2012-12-09 12:50:33,657::task::978::TaskManager.Task::(_decref) Task=`e1ec9f0e-87d6-49d1-afbd-e28aef0f99a1`::ref 0 aborting False
>Thread-2090::DEBUG::2012-12-09 12:50:33,669::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2090::DEBUG::2012-12-09 12:50:33,669::task::588::TaskManager.Task::(_updateState) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::moving from state init -> state preparing
>Thread-2090::INFO::2012-12-09 12:50:33,669::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2090::DEBUG::2012-12-09 12:50:33,670::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`36f21631-b77d-46b2-9083-20fed0972f16`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2090::DEBUG::2012-12-09 12:50:33,670::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2090::DEBUG::2012-12-09 12:50:33,670::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2090::DEBUG::2012-12-09 12:50:33,670::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`36f21631-b77d-46b2-9083-20fed0972f16`::Granted request
>Thread-2090::DEBUG::2012-12-09 12:50:33,671::task::817::TaskManager.Task::(resourceAcquired) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2090::DEBUG::2012-12-09 12:50:33,671::task::978::TaskManager.Task::(_decref) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::ref 1 aborting False
>Thread-2090::DEBUG::2012-12-09 12:50:33,671::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2090::DEBUG::2012-12-09 12:50:33,672::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2090::DEBUG::2012-12-09 12:50:33,770::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2090::WARNING::2012-12-09 12:50:33,771::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2090::DEBUG::2012-12-09 12:50:33,771::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2090::WARNING::2012-12-09 12:50:33,772::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2090::INFO::2012-12-09 12:50:33,772::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2090::DEBUG::2012-12-09 12:50:33,772::task::1172::TaskManager.Task::(prepare) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2090::DEBUG::2012-12-09 12:50:33,773::task::588::TaskManager.Task::(_updateState) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::moving from state preparing -> state finished
>Thread-2090::DEBUG::2012-12-09 12:50:33,773::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2090::DEBUG::2012-12-09 12:50:33,773::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2090::DEBUG::2012-12-09 12:50:33,773::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2090::DEBUG::2012-12-09 12:50:33,774::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2090::DEBUG::2012-12-09 12:50:33,774::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2090::DEBUG::2012-12-09 12:50:33,774::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2090::DEBUG::2012-12-09 12:50:33,774::task::978::TaskManager.Task::(_decref) Task=`22c106e3-1e48-44de-9f5d-d62eff26d4f9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:33,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:33,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155687 s, 65.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:35,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:36,024::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166153 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:38,031::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:38,072::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171065 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:40,079::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:40,119::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162738 s, 62.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:50:41,159::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:41,160::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:41,164::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:41,189::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000437471 s, 9.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:50:41,261::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:50:41,262::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:41,262::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:41,263::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2094::DEBUG::2012-12-09 12:50:41,382::task::588::TaskManager.Task::(_updateState) Task=`db68b8ca-5992-4ade-8bf3-e77d8e812ada`::moving from state init -> state preparing
>Thread-2094::INFO::2012-12-09 12:50:41,382::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2094::INFO::2012-12-09 12:50:41,382::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0248291492462', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-2094::DEBUG::2012-12-09 12:50:41,382::task::1172::TaskManager.Task::(prepare) Task=`db68b8ca-5992-4ade-8bf3-e77d8e812ada`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0248291492462', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-2094::DEBUG::2012-12-09 12:50:41,383::task::588::TaskManager.Task::(_updateState) Task=`db68b8ca-5992-4ade-8bf3-e77d8e812ada`::moving from state preparing -> state finished
>Thread-2094::DEBUG::2012-12-09 12:50:41,383::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2094::DEBUG::2012-12-09 12:50:41,383::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2094::DEBUG::2012-12-09 12:50:41,383::task::978::TaskManager.Task::(_decref) Task=`db68b8ca-5992-4ade-8bf3-e77d8e812ada`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:42,125::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:42,165::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162551 s, 63.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:44,172::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:44,211::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168488 s, 60.8 MB/s\n'; <rc> = 0
>Thread-2096::DEBUG::2012-12-09 12:50:44,901::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2096::DEBUG::2012-12-09 12:50:44,902::task::588::TaskManager.Task::(_updateState) Task=`0d1656d4-d334-41d3-8e25-c2d942e608f4`::moving from state init -> state preparing
>Thread-2096::INFO::2012-12-09 12:50:44,903::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2096::INFO::2012-12-09 12:50:44,903::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2096::DEBUG::2012-12-09 12:50:44,903::task::1172::TaskManager.Task::(prepare) Task=`0d1656d4-d334-41d3-8e25-c2d942e608f4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2096::DEBUG::2012-12-09 12:50:44,904::task::588::TaskManager.Task::(_updateState) Task=`0d1656d4-d334-41d3-8e25-c2d942e608f4`::moving from state preparing -> state finished
>Thread-2096::DEBUG::2012-12-09 12:50:44,904::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2096::DEBUG::2012-12-09 12:50:44,905::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2096::DEBUG::2012-12-09 12:50:44,905::task::978::TaskManager.Task::(_decref) Task=`0d1656d4-d334-41d3-8e25-c2d942e608f4`::ref 0 aborting False
>Thread-2097::DEBUG::2012-12-09 12:50:44,917::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2097::DEBUG::2012-12-09 12:50:44,918::task::588::TaskManager.Task::(_updateState) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::moving from state init -> state preparing
>Thread-2097::INFO::2012-12-09 12:50:44,919::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2097::DEBUG::2012-12-09 12:50:44,919::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`864be81e-094f-49c8-8b90-9adfcf826cbe`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2097::DEBUG::2012-12-09 12:50:44,919::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2097::DEBUG::2012-12-09 12:50:44,920::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2097::DEBUG::2012-12-09 12:50:44,920::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`864be81e-094f-49c8-8b90-9adfcf826cbe`::Granted request
>Thread-2097::DEBUG::2012-12-09 12:50:44,920::task::817::TaskManager.Task::(resourceAcquired) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2097::DEBUG::2012-12-09 12:50:44,920::task::978::TaskManager.Task::(_decref) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::ref 1 aborting False
>Thread-2097::DEBUG::2012-12-09 12:50:44,921::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2097::DEBUG::2012-12-09 12:50:44,922::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2097::DEBUG::2012-12-09 12:50:45,042::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2097::WARNING::2012-12-09 12:50:45,043::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2097::DEBUG::2012-12-09 12:50:45,043::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2097::WARNING::2012-12-09 12:50:45,044::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2097::INFO::2012-12-09 12:50:45,044::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2097::DEBUG::2012-12-09 12:50:45,044::task::1172::TaskManager.Task::(prepare) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2097::DEBUG::2012-12-09 12:50:45,045::task::588::TaskManager.Task::(_updateState) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::moving from state preparing -> state finished
>Thread-2097::DEBUG::2012-12-09 12:50:45,045::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2097::DEBUG::2012-12-09 12:50:45,045::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2097::DEBUG::2012-12-09 12:50:45,045::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2097::DEBUG::2012-12-09 12:50:45,046::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2097::DEBUG::2012-12-09 12:50:45,046::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2097::DEBUG::2012-12-09 12:50:45,046::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2097::DEBUG::2012-12-09 12:50:45,046::task::978::TaskManager.Task::(_decref) Task=`c0670be2-924b-47b5-b2d5-d764c409d1df`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:46,218::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:46,255::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158569 s, 64.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:48,262::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:48,304::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188173 s, 54.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:50,310::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:50,350::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169554 s, 60.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:50:51,196::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:50:51,207::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000550217 s, 7.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:50:51,264::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:50:51,264::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:50:51,363::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:50:51,364::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:50:51,364::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:50:51,365::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:50:52,357::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:52,402::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196795 s, 52.0 MB/s\n'; <rc> = 0
>Thread-2102::DEBUG::2012-12-09 12:50:53,212::task::588::TaskManager.Task::(_updateState) Task=`2d7db23f-e654-4202-800b-e0720c1989f3`::moving from state init -> state preparing
>Thread-2102::INFO::2012-12-09 12:50:53,213::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2102::INFO::2012-12-09 12:50:53,213::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113289356232', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-2102::DEBUG::2012-12-09 12:50:53,213::task::1172::TaskManager.Task::(prepare) Task=`2d7db23f-e654-4202-800b-e0720c1989f3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113289356232', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-2102::DEBUG::2012-12-09 12:50:53,213::task::588::TaskManager.Task::(_updateState) Task=`2d7db23f-e654-4202-800b-e0720c1989f3`::moving from state preparing -> state finished
>Thread-2102::DEBUG::2012-12-09 12:50:53,213::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2102::DEBUG::2012-12-09 12:50:53,214::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2102::DEBUG::2012-12-09 12:50:53,214::task::978::TaskManager.Task::(_decref) Task=`2d7db23f-e654-4202-800b-e0720c1989f3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:54,409::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:54,459::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02806 s, 36.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:50:56,466::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:56,509::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203425 s, 50.3 MB/s\n'; <rc> = 0
>Thread-2105::DEBUG::2012-12-09 12:50:58,038::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2105::DEBUG::2012-12-09 12:50:58,039::task::588::TaskManager.Task::(_updateState) Task=`d6dc43f8-f95e-48a8-b18b-56d47e9d02cc`::moving from state init -> state preparing
>Thread-2105::INFO::2012-12-09 12:50:58,039::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2105::INFO::2012-12-09 12:50:58,040::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2105::DEBUG::2012-12-09 12:50:58,040::task::1172::TaskManager.Task::(prepare) Task=`d6dc43f8-f95e-48a8-b18b-56d47e9d02cc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2105::DEBUG::2012-12-09 12:50:58,040::task::588::TaskManager.Task::(_updateState) Task=`d6dc43f8-f95e-48a8-b18b-56d47e9d02cc`::moving from state preparing -> state finished
>Thread-2105::DEBUG::2012-12-09 12:50:58,040::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2105::DEBUG::2012-12-09 12:50:58,040::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2105::DEBUG::2012-12-09 12:50:58,041::task::978::TaskManager.Task::(_decref) Task=`d6dc43f8-f95e-48a8-b18b-56d47e9d02cc`::ref 0 aborting False
>Thread-2106::DEBUG::2012-12-09 12:50:58,052::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2106::DEBUG::2012-12-09 12:50:58,052::task::588::TaskManager.Task::(_updateState) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::moving from state init -> state preparing
>Thread-2106::INFO::2012-12-09 12:50:58,052::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2106::DEBUG::2012-12-09 12:50:58,053::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f2399bf2-740c-4e00-9945-2c3233cf70df`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2106::DEBUG::2012-12-09 12:50:58,053::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2106::DEBUG::2012-12-09 12:50:58,053::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2106::DEBUG::2012-12-09 12:50:58,053::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f2399bf2-740c-4e00-9945-2c3233cf70df`::Granted request
>Thread-2106::DEBUG::2012-12-09 12:50:58,054::task::817::TaskManager.Task::(resourceAcquired) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2106::DEBUG::2012-12-09 12:50:58,054::task::978::TaskManager.Task::(_decref) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::ref 1 aborting False
>Thread-2106::DEBUG::2012-12-09 12:50:58,055::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2106::DEBUG::2012-12-09 12:50:58,055::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2106::DEBUG::2012-12-09 12:50:58,153::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2106::WARNING::2012-12-09 12:50:58,154::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2106::DEBUG::2012-12-09 12:50:58,154::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2106::WARNING::2012-12-09 12:50:58,155::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2106::INFO::2012-12-09 12:50:58,155::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2106::DEBUG::2012-12-09 12:50:58,155::task::1172::TaskManager.Task::(prepare) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2106::DEBUG::2012-12-09 12:50:58,156::task::588::TaskManager.Task::(_updateState) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::moving from state preparing -> state finished
>Thread-2106::DEBUG::2012-12-09 12:50:58,156::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2106::DEBUG::2012-12-09 12:50:58,156::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2106::DEBUG::2012-12-09 12:50:58,156::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2106::DEBUG::2012-12-09 12:50:58,156::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2106::DEBUG::2012-12-09 12:50:58,157::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2106::DEBUG::2012-12-09 12:50:58,157::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2106::DEBUG::2012-12-09 12:50:58,157::task::978::TaskManager.Task::(_decref) Task=`5f8303dc-7361-420e-af15-5c7ae3aaca9e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:50:58,516::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:50:58,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158285 s, 64.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:00,562::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:00,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168687 s, 60.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:01,209::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:01,217::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000373949 s, 11.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:51:01,366::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:51:01,366::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:51:01,476::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:01,477::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:01,477::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:01,478::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:02,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:02,650::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158954 s, 64.4 MB/s\n'; <rc> = 0
>Thread-2110::DEBUG::2012-12-09 12:51:04,551::task::588::TaskManager.Task::(_updateState) Task=`f979418d-55a1-4004-abb8-54dc19607ca3`::moving from state init -> state preparing
>Thread-2110::INFO::2012-12-09 12:51:04,551::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2110::INFO::2012-12-09 12:51:04,551::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00767993927002', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2110::DEBUG::2012-12-09 12:51:04,552::task::1172::TaskManager.Task::(prepare) Task=`f979418d-55a1-4004-abb8-54dc19607ca3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00767993927002', 'lastCheck': '3.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2110::DEBUG::2012-12-09 12:51:04,552::task::588::TaskManager.Task::(_updateState) Task=`f979418d-55a1-4004-abb8-54dc19607ca3`::moving from state preparing -> state finished
>Thread-2110::DEBUG::2012-12-09 12:51:04,552::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2110::DEBUG::2012-12-09 12:51:04,552::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2110::DEBUG::2012-12-09 12:51:04,552::task::978::TaskManager.Task::(_decref) Task=`f979418d-55a1-4004-abb8-54dc19607ca3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:04,657::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:04,698::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166258 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:06,704::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:06,743::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162958 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:08,750::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:08,789::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159279 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:10,795::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:10,848::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210168 s, 48.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:11,220::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:11,228::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000467404 s, 8.8 MB/s\n'; <rc> = 0
>Thread-2114::DEBUG::2012-12-09 12:51:11,451::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2114::DEBUG::2012-12-09 12:51:11,452::task::588::TaskManager.Task::(_updateState) Task=`0e50adbd-2782-4a6f-b45b-d26b589994ff`::moving from state init -> state preparing
>Thread-2114::INFO::2012-12-09 12:51:11,452::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2114::INFO::2012-12-09 12:51:11,452::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2114::DEBUG::2012-12-09 12:51:11,452::task::1172::TaskManager.Task::(prepare) Task=`0e50adbd-2782-4a6f-b45b-d26b589994ff`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2114::DEBUG::2012-12-09 12:51:11,452::task::588::TaskManager.Task::(_updateState) Task=`0e50adbd-2782-4a6f-b45b-d26b589994ff`::moving from state preparing -> state finished
>Thread-2114::DEBUG::2012-12-09 12:51:11,453::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2114::DEBUG::2012-12-09 12:51:11,453::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2114::DEBUG::2012-12-09 12:51:11,453::task::978::TaskManager.Task::(_decref) Task=`0e50adbd-2782-4a6f-b45b-d26b589994ff`::ref 0 aborting False
>Thread-2115::DEBUG::2012-12-09 12:51:11,466::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2115::DEBUG::2012-12-09 12:51:11,467::task::588::TaskManager.Task::(_updateState) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::moving from state init -> state preparing
>Thread-2115::INFO::2012-12-09 12:51:11,467::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2115::DEBUG::2012-12-09 12:51:11,468::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7ca30059-e94f-4daf-894d-aa3458ca1389`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2115::DEBUG::2012-12-09 12:51:11,468::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2115::DEBUG::2012-12-09 12:51:11,468::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2115::DEBUG::2012-12-09 12:51:11,468::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7ca30059-e94f-4daf-894d-aa3458ca1389`::Granted request
>Thread-2115::DEBUG::2012-12-09 12:51:11,469::task::817::TaskManager.Task::(resourceAcquired) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2115::DEBUG::2012-12-09 12:51:11,469::task::978::TaskManager.Task::(_decref) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::ref 1 aborting False
>Thread-2115::DEBUG::2012-12-09 12:51:11,470::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2115::DEBUG::2012-12-09 12:51:11,470::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2115::DEBUG::2012-12-09 12:51:11,566::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2115::WARNING::2012-12-09 12:51:11,567::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2115::DEBUG::2012-12-09 12:51:11,567::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2115::WARNING::2012-12-09 12:51:11,568::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-51::DEBUG::2012-12-09 12:51:11,568::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2115::INFO::2012-12-09 12:51:11,569::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-51::DEBUG::2012-12-09 12:51:11,569::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2115::DEBUG::2012-12-09 12:51:11,570::task::1172::TaskManager.Task::(prepare) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2115::DEBUG::2012-12-09 12:51:11,575::task::588::TaskManager.Task::(_updateState) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::moving from state preparing -> state finished
>Thread-2115::DEBUG::2012-12-09 12:51:11,575::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2115::DEBUG::2012-12-09 12:51:11,575::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2115::DEBUG::2012-12-09 12:51:11,576::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2115::DEBUG::2012-12-09 12:51:11,576::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2115::DEBUG::2012-12-09 12:51:11,576::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2115::DEBUG::2012-12-09 12:51:11,576::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2115::DEBUG::2012-12-09 12:51:11,577::task::978::TaskManager.Task::(_decref) Task=`59a87b0d-d554-4f11-9e70-fa07546e6a98`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:51:11,666::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:11,667::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:11,667::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:11,667::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:12,854::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:12,900::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221859 s, 46.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:14,907::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:14,948::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174305 s, 58.7 MB/s\n'; <rc> = 0
>Thread-2118::DEBUG::2012-12-09 12:51:16,583::task::588::TaskManager.Task::(_updateState) Task=`a10f3fc6-de8f-4df5-b839-868ca80637aa`::moving from state init -> state preparing
>Thread-2118::INFO::2012-12-09 12:51:16,583::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2118::INFO::2012-12-09 12:51:16,583::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00882506370544', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2118::DEBUG::2012-12-09 12:51:16,584::task::1172::TaskManager.Task::(prepare) Task=`a10f3fc6-de8f-4df5-b839-868ca80637aa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00882506370544', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2118::DEBUG::2012-12-09 12:51:16,584::task::588::TaskManager.Task::(_updateState) Task=`a10f3fc6-de8f-4df5-b839-868ca80637aa`::moving from state preparing -> state finished
>Thread-2118::DEBUG::2012-12-09 12:51:16,584::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2118::DEBUG::2012-12-09 12:51:16,584::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2118::DEBUG::2012-12-09 12:51:16,584::task::978::TaskManager.Task::(_decref) Task=`a10f3fc6-de8f-4df5-b839-868ca80637aa`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:16,955::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:16,996::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164295 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:19,002::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:19,045::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016938 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:21,052::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:21,091::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163962 s, 62.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:21,231::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:21,239::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000500002 s, 8.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:51:21,668::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:51:21,669::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:51:21,767::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:21,768::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:21,768::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:21,769::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:23,098::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:23,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165633 s, 61.8 MB/s\n'; <rc> = 0
>Thread-2123::DEBUG::2012-12-09 12:51:24,764::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2123::DEBUG::2012-12-09 12:51:24,765::task::588::TaskManager.Task::(_updateState) Task=`1d4a3c3a-8a46-40cd-801d-1e4fc2183817`::moving from state init -> state preparing
>Thread-2123::INFO::2012-12-09 12:51:24,765::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2123::INFO::2012-12-09 12:51:24,765::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2123::DEBUG::2012-12-09 12:51:24,765::task::1172::TaskManager.Task::(prepare) Task=`1d4a3c3a-8a46-40cd-801d-1e4fc2183817`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2123::DEBUG::2012-12-09 12:51:24,766::task::588::TaskManager.Task::(_updateState) Task=`1d4a3c3a-8a46-40cd-801d-1e4fc2183817`::moving from state preparing -> state finished
>Thread-2123::DEBUG::2012-12-09 12:51:24,766::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2123::DEBUG::2012-12-09 12:51:24,766::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2123::DEBUG::2012-12-09 12:51:24,766::task::978::TaskManager.Task::(_decref) Task=`1d4a3c3a-8a46-40cd-801d-1e4fc2183817`::ref 0 aborting False
>Thread-2124::DEBUG::2012-12-09 12:51:24,778::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2124::DEBUG::2012-12-09 12:51:24,779::task::588::TaskManager.Task::(_updateState) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::moving from state init -> state preparing
>Thread-2124::INFO::2012-12-09 12:51:24,779::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2124::DEBUG::2012-12-09 12:51:24,779::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f30a2fb1-f1a4-4be9-835d-2b9dab1ca024`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2124::DEBUG::2012-12-09 12:51:24,780::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2124::DEBUG::2012-12-09 12:51:24,780::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2124::DEBUG::2012-12-09 12:51:24,780::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f30a2fb1-f1a4-4be9-835d-2b9dab1ca024`::Granted request
>Thread-2124::DEBUG::2012-12-09 12:51:24,780::task::817::TaskManager.Task::(resourceAcquired) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2124::DEBUG::2012-12-09 12:51:24,781::task::978::TaskManager.Task::(_decref) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::ref 1 aborting False
>Thread-2124::DEBUG::2012-12-09 12:51:24,781::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2124::DEBUG::2012-12-09 12:51:24,782::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2124::DEBUG::2012-12-09 12:51:24,880::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2124::WARNING::2012-12-09 12:51:24,881::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2124::DEBUG::2012-12-09 12:51:24,881::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2124::WARNING::2012-12-09 12:51:24,881::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2124::INFO::2012-12-09 12:51:24,882::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2124::DEBUG::2012-12-09 12:51:24,882::task::1172::TaskManager.Task::(prepare) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2124::DEBUG::2012-12-09 12:51:24,882::task::588::TaskManager.Task::(_updateState) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::moving from state preparing -> state finished
>Thread-2124::DEBUG::2012-12-09 12:51:24,883::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2124::DEBUG::2012-12-09 12:51:24,883::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2124::DEBUG::2012-12-09 12:51:24,883::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2124::DEBUG::2012-12-09 12:51:24,883::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2124::DEBUG::2012-12-09 12:51:24,883::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2124::DEBUG::2012-12-09 12:51:24,884::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2124::DEBUG::2012-12-09 12:51:24,884::task::978::TaskManager.Task::(_decref) Task=`af121ed8-c2db-4efc-b531-cabfe13ca194`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:25,143::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:25,182::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016201 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:27,188::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:27,228::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171291 s, 59.8 MB/s\n'; <rc> = 0
>Thread-2126::DEBUG::2012-12-09 12:51:27,878::task::588::TaskManager.Task::(_updateState) Task=`0bccba66-916b-4ed3-93eb-1c45fa34f68b`::moving from state init -> state preparing
>Thread-2126::INFO::2012-12-09 12:51:27,879::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2126::INFO::2012-12-09 12:51:27,879::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00778794288635', 'lastCheck': '6.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-2126::DEBUG::2012-12-09 12:51:27,879::task::1172::TaskManager.Task::(prepare) Task=`0bccba66-916b-4ed3-93eb-1c45fa34f68b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00778794288635', 'lastCheck': '6.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-2126::DEBUG::2012-12-09 12:51:27,879::task::588::TaskManager.Task::(_updateState) Task=`0bccba66-916b-4ed3-93eb-1c45fa34f68b`::moving from state preparing -> state finished
>Thread-2126::DEBUG::2012-12-09 12:51:27,879::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2126::DEBUG::2012-12-09 12:51:27,880::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2126::DEBUG::2012-12-09 12:51:27,880::task::978::TaskManager.Task::(_decref) Task=`0bccba66-916b-4ed3-93eb-1c45fa34f68b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:29,235::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:29,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167746 s, 61.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:31,242::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:31,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00082735 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:31,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:31,329::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207471 s, 49.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:51:31,769::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:51:31,770::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:51:31,869::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:31,870::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:31,871::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:31,871::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:33,336::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:33,376::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184378 s, 55.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:35,383::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:35,423::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167738 s, 61.0 MB/s\n'; <rc> = 0
>Thread-2132::DEBUG::2012-12-09 12:51:36,478::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2132::DEBUG::2012-12-09 12:51:36,478::task::588::TaskManager.Task::(_updateState) Task=`7e97dc71-1bc9-475d-b268-8d776e6e58cc`::moving from state init -> state preparing
>Thread-2132::INFO::2012-12-09 12:51:36,478::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2132::INFO::2012-12-09 12:51:36,479::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2132::DEBUG::2012-12-09 12:51:36,479::task::1172::TaskManager.Task::(prepare) Task=`7e97dc71-1bc9-475d-b268-8d776e6e58cc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2132::DEBUG::2012-12-09 12:51:36,479::task::588::TaskManager.Task::(_updateState) Task=`7e97dc71-1bc9-475d-b268-8d776e6e58cc`::moving from state preparing -> state finished
>Thread-2132::DEBUG::2012-12-09 12:51:36,479::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2132::DEBUG::2012-12-09 12:51:36,479::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2132::DEBUG::2012-12-09 12:51:36,480::task::978::TaskManager.Task::(_decref) Task=`7e97dc71-1bc9-475d-b268-8d776e6e58cc`::ref 0 aborting False
>Thread-2133::DEBUG::2012-12-09 12:51:36,492::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2133::DEBUG::2012-12-09 12:51:36,492::task::588::TaskManager.Task::(_updateState) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::moving from state init -> state preparing
>Thread-2133::INFO::2012-12-09 12:51:36,493::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2133::DEBUG::2012-12-09 12:51:36,493::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51e146de-87d0-4e48-8ce1-f08be383a9a4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2133::DEBUG::2012-12-09 12:51:36,493::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2133::DEBUG::2012-12-09 12:51:36,494::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2133::DEBUG::2012-12-09 12:51:36,494::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51e146de-87d0-4e48-8ce1-f08be383a9a4`::Granted request
>Thread-2133::DEBUG::2012-12-09 12:51:36,494::task::817::TaskManager.Task::(resourceAcquired) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2133::DEBUG::2012-12-09 12:51:36,494::task::978::TaskManager.Task::(_decref) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::ref 1 aborting False
>Thread-2133::DEBUG::2012-12-09 12:51:36,495::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2133::DEBUG::2012-12-09 12:51:36,495::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2133::DEBUG::2012-12-09 12:51:36,594::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2133::WARNING::2012-12-09 12:51:36,595::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2133::DEBUG::2012-12-09 12:51:36,596::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2133::WARNING::2012-12-09 12:51:36,596::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2133::INFO::2012-12-09 12:51:36,596::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2133::DEBUG::2012-12-09 12:51:36,597::task::1172::TaskManager.Task::(prepare) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2133::DEBUG::2012-12-09 12:51:36,597::task::588::TaskManager.Task::(_updateState) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::moving from state preparing -> state finished
>Thread-2133::DEBUG::2012-12-09 12:51:36,597::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2133::DEBUG::2012-12-09 12:51:36,597::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2133::DEBUG::2012-12-09 12:51:36,598::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2133::DEBUG::2012-12-09 12:51:36,598::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2133::DEBUG::2012-12-09 12:51:36,598::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2133::DEBUG::2012-12-09 12:51:36,598::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2133::DEBUG::2012-12-09 12:51:36,599::task::978::TaskManager.Task::(_decref) Task=`2a18be21-a991-4501-90fb-e48c22b81410`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:37,431::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:37,473::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188152 s, 54.4 MB/s\n'; <rc> = 0
>Thread-2134::DEBUG::2012-12-09 12:51:38,427::task::588::TaskManager.Task::(_updateState) Task=`7faecf69-8b44-41df-96dc-7392f49b1c52`::moving from state init -> state preparing
>Thread-2134::INFO::2012-12-09 12:51:38,427::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2134::INFO::2012-12-09 12:51:38,427::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137391090393', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2134::DEBUG::2012-12-09 12:51:38,428::task::1172::TaskManager.Task::(prepare) Task=`7faecf69-8b44-41df-96dc-7392f49b1c52`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137391090393', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2134::DEBUG::2012-12-09 12:51:38,428::task::588::TaskManager.Task::(_updateState) Task=`7faecf69-8b44-41df-96dc-7392f49b1c52`::moving from state preparing -> state finished
>Thread-2134::DEBUG::2012-12-09 12:51:38,428::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2134::DEBUG::2012-12-09 12:51:38,429::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2134::DEBUG::2012-12-09 12:51:38,429::task::978::TaskManager.Task::(_decref) Task=`7faecf69-8b44-41df-96dc-7392f49b1c52`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:39,479::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:39,518::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152822 s, 67.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:41,260::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:41,268::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0011262 s, 3.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:41,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:41,562::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161582 s, 63.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:51:41,872::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:51:41,873::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:51:41,967::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:41,968::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:41,969::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:41,969::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:43,569::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:43,610::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017576 s, 58.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:45,617::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:45,660::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166195 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:47,667::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:47,707::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165807 s, 61.8 MB/s\n'; <rc> = 0
>Thread-2140::DEBUG::2012-12-09 12:51:48,547::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2140::DEBUG::2012-12-09 12:51:48,547::task::588::TaskManager.Task::(_updateState) Task=`dfce945f-da1f-4c01-8dba-d3c38b4adedd`::moving from state init -> state preparing
>Thread-2140::INFO::2012-12-09 12:51:48,548::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2140::INFO::2012-12-09 12:51:48,548::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2140::DEBUG::2012-12-09 12:51:48,548::task::1172::TaskManager.Task::(prepare) Task=`dfce945f-da1f-4c01-8dba-d3c38b4adedd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2140::DEBUG::2012-12-09 12:51:48,548::task::588::TaskManager.Task::(_updateState) Task=`dfce945f-da1f-4c01-8dba-d3c38b4adedd`::moving from state preparing -> state finished
>Thread-2140::DEBUG::2012-12-09 12:51:48,549::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2140::DEBUG::2012-12-09 12:51:48,549::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2140::DEBUG::2012-12-09 12:51:48,549::task::978::TaskManager.Task::(_decref) Task=`dfce945f-da1f-4c01-8dba-d3c38b4adedd`::ref 0 aborting False
>Thread-2141::DEBUG::2012-12-09 12:51:48,561::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2141::DEBUG::2012-12-09 12:51:48,561::task::588::TaskManager.Task::(_updateState) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::moving from state init -> state preparing
>Thread-2141::INFO::2012-12-09 12:51:48,561::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2141::DEBUG::2012-12-09 12:51:48,562::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e38410c9-c744-4bf2-8a93-f0961aa8ab21`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2141::DEBUG::2012-12-09 12:51:48,562::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2141::DEBUG::2012-12-09 12:51:48,562::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2141::DEBUG::2012-12-09 12:51:48,562::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e38410c9-c744-4bf2-8a93-f0961aa8ab21`::Granted request
>Thread-2141::DEBUG::2012-12-09 12:51:48,563::task::817::TaskManager.Task::(resourceAcquired) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2141::DEBUG::2012-12-09 12:51:48,563::task::978::TaskManager.Task::(_decref) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::ref 1 aborting False
>Thread-2141::DEBUG::2012-12-09 12:51:48,563::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2141::DEBUG::2012-12-09 12:51:48,564::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2141::DEBUG::2012-12-09 12:51:48,665::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2141::WARNING::2012-12-09 12:51:48,666::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2141::DEBUG::2012-12-09 12:51:48,666::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2141::WARNING::2012-12-09 12:51:48,667::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2141::INFO::2012-12-09 12:51:48,667::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2141::DEBUG::2012-12-09 12:51:48,667::task::1172::TaskManager.Task::(prepare) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2141::DEBUG::2012-12-09 12:51:48,667::task::588::TaskManager.Task::(_updateState) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::moving from state preparing -> state finished
>Thread-2141::DEBUG::2012-12-09 12:51:48,668::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2141::DEBUG::2012-12-09 12:51:48,668::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2141::DEBUG::2012-12-09 12:51:48,668::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2141::DEBUG::2012-12-09 12:51:48,668::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2141::DEBUG::2012-12-09 12:51:48,669::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2141::DEBUG::2012-12-09 12:51:48,669::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2141::DEBUG::2012-12-09 12:51:48,669::task::978::TaskManager.Task::(_decref) Task=`2fc9af49-f2d2-4a2c-b553-3b68efd045c1`::ref 0 aborting False
>Thread-2142::DEBUG::2012-12-09 12:51:49,372::task::588::TaskManager.Task::(_updateState) Task=`76d5520b-2d2d-4fdf-8904-514cf6f8ac21`::moving from state init -> state preparing
>Thread-2142::INFO::2012-12-09 12:51:49,373::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2142::INFO::2012-12-09 12:51:49,373::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00864696502686', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-2142::DEBUG::2012-12-09 12:51:49,374::task::1172::TaskManager.Task::(prepare) Task=`76d5520b-2d2d-4fdf-8904-514cf6f8ac21`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00864696502686', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.4', 'code': 358, 'valid': False}}
>Thread-2142::DEBUG::2012-12-09 12:51:49,374::task::588::TaskManager.Task::(_updateState) Task=`76d5520b-2d2d-4fdf-8904-514cf6f8ac21`::moving from state preparing -> state finished
>Thread-2142::DEBUG::2012-12-09 12:51:49,374::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2142::DEBUG::2012-12-09 12:51:49,375::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2142::DEBUG::2012-12-09 12:51:49,375::task::978::TaskManager.Task::(_decref) Task=`76d5520b-2d2d-4fdf-8904-514cf6f8ac21`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:51:49,714::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:49,753::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156823 s, 65.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:51:51,272::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:51:51,283::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000774151 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:51,760::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:51,807::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212859 s, 48.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:51:51,970::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:51:51,971::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:51:52,073::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:51:52,074::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:51:52,074::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:51:52,074::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:51:53,814::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:53,853::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153535 s, 66.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:55,861::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:55,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171601 s, 59.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:57,912::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:57,950::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159559 s, 64.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:51:59,957::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:51:59,995::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152509 s, 67.1 MB/s\n'; <rc> = 0
>Thread-2148::DEBUG::2012-12-09 12:52:00,693::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2148::DEBUG::2012-12-09 12:52:00,694::task::588::TaskManager.Task::(_updateState) Task=`da665875-aa32-4461-a714-e092bf408fc5`::moving from state init -> state preparing
>Thread-2148::INFO::2012-12-09 12:52:00,695::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2148::INFO::2012-12-09 12:52:00,695::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2148::DEBUG::2012-12-09 12:52:00,696::task::1172::TaskManager.Task::(prepare) Task=`da665875-aa32-4461-a714-e092bf408fc5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2148::DEBUG::2012-12-09 12:52:00,696::task::588::TaskManager.Task::(_updateState) Task=`da665875-aa32-4461-a714-e092bf408fc5`::moving from state preparing -> state finished
>Thread-2148::DEBUG::2012-12-09 12:52:00,696::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2148::DEBUG::2012-12-09 12:52:00,697::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2148::DEBUG::2012-12-09 12:52:00,697::task::978::TaskManager.Task::(_decref) Task=`da665875-aa32-4461-a714-e092bf408fc5`::ref 0 aborting False
>Thread-2149::DEBUG::2012-12-09 12:52:00,711::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2149::DEBUG::2012-12-09 12:52:00,711::task::588::TaskManager.Task::(_updateState) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::moving from state init -> state preparing
>Thread-2149::INFO::2012-12-09 12:52:00,711::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2149::DEBUG::2012-12-09 12:52:00,712::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bb2dde7a-1b8a-4839-8be9-dd64358ee2fa`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2149::DEBUG::2012-12-09 12:52:00,712::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2149::DEBUG::2012-12-09 12:52:00,712::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2149::DEBUG::2012-12-09 12:52:00,712::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bb2dde7a-1b8a-4839-8be9-dd64358ee2fa`::Granted request
>Thread-2149::DEBUG::2012-12-09 12:52:00,713::task::817::TaskManager.Task::(resourceAcquired) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2149::DEBUG::2012-12-09 12:52:00,713::task::978::TaskManager.Task::(_decref) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::ref 1 aborting False
>Thread-2149::DEBUG::2012-12-09 12:52:00,714::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2149::DEBUG::2012-12-09 12:52:00,714::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2149::DEBUG::2012-12-09 12:52:00,808::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2149::WARNING::2012-12-09 12:52:00,809::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2149::DEBUG::2012-12-09 12:52:00,809::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2149::WARNING::2012-12-09 12:52:00,810::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2149::INFO::2012-12-09 12:52:00,810::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2149::DEBUG::2012-12-09 12:52:00,810::task::1172::TaskManager.Task::(prepare) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2149::DEBUG::2012-12-09 12:52:00,811::task::588::TaskManager.Task::(_updateState) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::moving from state preparing -> state finished
>Thread-2149::DEBUG::2012-12-09 12:52:00,811::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2149::DEBUG::2012-12-09 12:52:00,811::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2149::DEBUG::2012-12-09 12:52:00,811::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2149::DEBUG::2012-12-09 12:52:00,811::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2149::DEBUG::2012-12-09 12:52:00,812::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2149::DEBUG::2012-12-09 12:52:00,812::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2149::DEBUG::2012-12-09 12:52:00,812::task::978::TaskManager.Task::(_decref) Task=`5a56fbf9-6d53-4aec-a8a6-f31c0a7d3593`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:52:01,289::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:01,300::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000388793 s, 10.5 MB/s\n'; <rc> = 0
>Thread-2150::DEBUG::2012-12-09 12:52:01,747::task::588::TaskManager.Task::(_updateState) Task=`a7a016e1-7701-4c8a-b3f9-c0a414c8a6f3`::moving from state init -> state preparing
>Thread-2150::INFO::2012-12-09 12:52:01,748::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2150::INFO::2012-12-09 12:52:01,748::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113170146942', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-2150::DEBUG::2012-12-09 12:52:01,748::task::1172::TaskManager.Task::(prepare) Task=`a7a016e1-7701-4c8a-b3f9-c0a414c8a6f3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113170146942', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-2150::DEBUG::2012-12-09 12:52:01,748::task::588::TaskManager.Task::(_updateState) Task=`a7a016e1-7701-4c8a-b3f9-c0a414c8a6f3`::moving from state preparing -> state finished
>Thread-2150::DEBUG::2012-12-09 12:52:01,749::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2150::DEBUG::2012-12-09 12:52:01,749::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2150::DEBUG::2012-12-09 12:52:01,749::task::978::TaskManager.Task::(_decref) Task=`a7a016e1-7701-4c8a-b3f9-c0a414c8a6f3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:52:02,002::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:02,041::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160637 s, 63.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:02,075::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:02,076::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:02,175::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:02,176::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:02,176::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:02,177::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:04,048::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:04,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171699 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:06,094::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:06,131::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154238 s, 66.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:08,138::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:08,178::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175439 s, 58.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:10,185::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:10,229::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202129 s, 50.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:52:11,303::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:11,310::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000410102 s, 10.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:12,178::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:12,178::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:12,235::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:12,284::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:12,286::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:12,286::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:12,286::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:12,287::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225207 s, 45.5 MB/s\n'; <rc> = 0
>Thread-2156::DEBUG::2012-12-09 12:52:12,742::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2156::DEBUG::2012-12-09 12:52:12,743::task::588::TaskManager.Task::(_updateState) Task=`14155041-e85e-429c-995e-2f60e699d842`::moving from state init -> state preparing
>Thread-2156::INFO::2012-12-09 12:52:12,743::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2156::INFO::2012-12-09 12:52:12,743::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2156::DEBUG::2012-12-09 12:52:12,744::task::1172::TaskManager.Task::(prepare) Task=`14155041-e85e-429c-995e-2f60e699d842`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2156::DEBUG::2012-12-09 12:52:12,744::task::588::TaskManager.Task::(_updateState) Task=`14155041-e85e-429c-995e-2f60e699d842`::moving from state preparing -> state finished
>Thread-2156::DEBUG::2012-12-09 12:52:12,744::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2156::DEBUG::2012-12-09 12:52:12,744::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2156::DEBUG::2012-12-09 12:52:12,744::task::978::TaskManager.Task::(_decref) Task=`14155041-e85e-429c-995e-2f60e699d842`::ref 0 aborting False
>Thread-2157::DEBUG::2012-12-09 12:52:12,756::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2157::DEBUG::2012-12-09 12:52:12,757::task::588::TaskManager.Task::(_updateState) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::moving from state init -> state preparing
>Thread-2157::INFO::2012-12-09 12:52:12,757::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2157::DEBUG::2012-12-09 12:52:12,758::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c8021d2f-4d7a-4d0d-b97e-b282f8730583`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2157::DEBUG::2012-12-09 12:52:12,758::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2157::DEBUG::2012-12-09 12:52:12,758::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2157::DEBUG::2012-12-09 12:52:12,758::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c8021d2f-4d7a-4d0d-b97e-b282f8730583`::Granted request
>Thread-2157::DEBUG::2012-12-09 12:52:12,758::task::817::TaskManager.Task::(resourceAcquired) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2157::DEBUG::2012-12-09 12:52:12,759::task::978::TaskManager.Task::(_decref) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::ref 1 aborting False
>Thread-2157::DEBUG::2012-12-09 12:52:12,759::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2157::DEBUG::2012-12-09 12:52:12,760::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2157::DEBUG::2012-12-09 12:52:12,863::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2157::WARNING::2012-12-09 12:52:12,864::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2157::DEBUG::2012-12-09 12:52:12,864::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2157::WARNING::2012-12-09 12:52:12,865::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2157::INFO::2012-12-09 12:52:12,865::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2157::DEBUG::2012-12-09 12:52:12,865::task::1172::TaskManager.Task::(prepare) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2157::DEBUG::2012-12-09 12:52:12,866::task::588::TaskManager.Task::(_updateState) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::moving from state preparing -> state finished
>Thread-2157::DEBUG::2012-12-09 12:52:12,866::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2157::DEBUG::2012-12-09 12:52:12,866::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2157::DEBUG::2012-12-09 12:52:12,866::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2157::DEBUG::2012-12-09 12:52:12,866::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2157::DEBUG::2012-12-09 12:52:12,867::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2157::DEBUG::2012-12-09 12:52:12,867::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2157::DEBUG::2012-12-09 12:52:12,867::task::978::TaskManager.Task::(_decref) Task=`6f32b081-176c-45d9-b6f0-9e3c92ca3501`::ref 0 aborting False
>Thread-2158::DEBUG::2012-12-09 12:52:13,723::task::588::TaskManager.Task::(_updateState) Task=`02f5dae5-c3a8-45d2-8e78-7802038f67fa`::moving from state init -> state preparing
>Thread-2158::INFO::2012-12-09 12:52:13,724::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2158::INFO::2012-12-09 12:52:13,724::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00784301757812', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-2158::DEBUG::2012-12-09 12:52:13,724::task::1172::TaskManager.Task::(prepare) Task=`02f5dae5-c3a8-45d2-8e78-7802038f67fa`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00784301757812', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-2158::DEBUG::2012-12-09 12:52:13,725::task::588::TaskManager.Task::(_updateState) Task=`02f5dae5-c3a8-45d2-8e78-7802038f67fa`::moving from state preparing -> state finished
>Thread-2158::DEBUG::2012-12-09 12:52:13,725::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2158::DEBUG::2012-12-09 12:52:13,725::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2158::DEBUG::2012-12-09 12:52:13,725::task::978::TaskManager.Task::(_decref) Task=`02f5dae5-c3a8-45d2-8e78-7802038f67fa`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:52:14,294::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:14,343::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153873 s, 66.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:16,349::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:16,391::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181432 s, 56.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:18,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:18,437::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162723 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:20,443::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:20,488::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175186 s, 58.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:52:21,313::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:21,324::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000396645 s, 10.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:22,287::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:22,288::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:22,382::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:22,384::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:22,384::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:22,384::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:22,495::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:22,537::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155124 s, 66.0 MB/s\n'; <rc> = 0
>Thread-2164::DEBUG::2012-12-09 12:52:23,688::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2164::DEBUG::2012-12-09 12:52:23,689::task::588::TaskManager.Task::(_updateState) Task=`d84a40ba-877d-4cc0-a7df-02d79ab5428e`::moving from state init -> state preparing
>Thread-2164::INFO::2012-12-09 12:52:23,689::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2164::INFO::2012-12-09 12:52:23,689::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2164::DEBUG::2012-12-09 12:52:23,689::task::1172::TaskManager.Task::(prepare) Task=`d84a40ba-877d-4cc0-a7df-02d79ab5428e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2164::DEBUG::2012-12-09 12:52:23,690::task::588::TaskManager.Task::(_updateState) Task=`d84a40ba-877d-4cc0-a7df-02d79ab5428e`::moving from state preparing -> state finished
>Thread-2164::DEBUG::2012-12-09 12:52:23,690::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2164::DEBUG::2012-12-09 12:52:23,690::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2164::DEBUG::2012-12-09 12:52:23,690::task::978::TaskManager.Task::(_decref) Task=`d84a40ba-877d-4cc0-a7df-02d79ab5428e`::ref 0 aborting False
>Thread-2165::DEBUG::2012-12-09 12:52:23,706::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2165::DEBUG::2012-12-09 12:52:23,707::task::588::TaskManager.Task::(_updateState) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::moving from state init -> state preparing
>Thread-2165::INFO::2012-12-09 12:52:23,707::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2165::DEBUG::2012-12-09 12:52:23,708::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8857d286-f629-46ab-8562-8ebddf65b314`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2165::DEBUG::2012-12-09 12:52:23,708::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2165::DEBUG::2012-12-09 12:52:23,708::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2165::DEBUG::2012-12-09 12:52:23,708::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8857d286-f629-46ab-8562-8ebddf65b314`::Granted request
>Thread-2165::DEBUG::2012-12-09 12:52:23,708::task::817::TaskManager.Task::(resourceAcquired) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2165::DEBUG::2012-12-09 12:52:23,709::task::978::TaskManager.Task::(_decref) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::ref 1 aborting False
>Thread-2165::DEBUG::2012-12-09 12:52:23,709::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2165::DEBUG::2012-12-09 12:52:23,710::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2165::DEBUG::2012-12-09 12:52:23,807::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2165::WARNING::2012-12-09 12:52:23,808::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2165::DEBUG::2012-12-09 12:52:23,808::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2165::WARNING::2012-12-09 12:52:23,809::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2165::INFO::2012-12-09 12:52:23,809::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2165::DEBUG::2012-12-09 12:52:23,809::task::1172::TaskManager.Task::(prepare) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2165::DEBUG::2012-12-09 12:52:23,810::task::588::TaskManager.Task::(_updateState) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::moving from state preparing -> state finished
>Thread-2165::DEBUG::2012-12-09 12:52:23,810::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2165::DEBUG::2012-12-09 12:52:23,810::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2165::DEBUG::2012-12-09 12:52:23,810::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2165::DEBUG::2012-12-09 12:52:23,810::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2165::DEBUG::2012-12-09 12:52:23,811::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2165::DEBUG::2012-12-09 12:52:23,811::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2165::DEBUG::2012-12-09 12:52:23,811::task::978::TaskManager.Task::(_decref) Task=`3d08e44b-97aa-484c-a835-f83b9b844d19`::ref 0 aborting False
>Thread-2166::DEBUG::2012-12-09 12:52:24,375::task::588::TaskManager.Task::(_updateState) Task=`972c858f-cead-4470-9049-5996d0bc7215`::moving from state init -> state preparing
>Thread-2166::INFO::2012-12-09 12:52:24,376::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2166::INFO::2012-12-09 12:52:24,376::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011626958847', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-2166::DEBUG::2012-12-09 12:52:24,376::task::1172::TaskManager.Task::(prepare) Task=`972c858f-cead-4470-9049-5996d0bc7215`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011626958847', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-2166::DEBUG::2012-12-09 12:52:24,376::task::588::TaskManager.Task::(_updateState) Task=`972c858f-cead-4470-9049-5996d0bc7215`::moving from state preparing -> state finished
>Thread-2166::DEBUG::2012-12-09 12:52:24,376::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2166::DEBUG::2012-12-09 12:52:24,377::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2166::DEBUG::2012-12-09 12:52:24,377::task::978::TaskManager.Task::(_decref) Task=`972c858f-cead-4470-9049-5996d0bc7215`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:52:24,544::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:24,585::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174034 s, 58.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:26,591::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:26,629::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164339 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:28,636::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:28,677::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163582 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:30,684::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:30,724::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173174 s, 59.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:52:31,327::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:31,338::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000423009 s, 9.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:32,385::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:32,386::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:32,482::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:32,483::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:32,484::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:32,484::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:32,731::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:32,776::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219689 s, 46.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:34,783::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:34,856::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322946 s, 31.7 MB/s\n'; <rc> = 0
>Thread-2172::DEBUG::2012-12-09 12:52:35,722::task::588::TaskManager.Task::(_updateState) Task=`03c253fa-d100-40e7-8870-b195ceac0e00`::moving from state init -> state preparing
>Thread-2172::INFO::2012-12-09 12:52:35,722::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2172::INFO::2012-12-09 12:52:35,723::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011873960495', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-2172::DEBUG::2012-12-09 12:52:35,723::task::1172::TaskManager.Task::(prepare) Task=`03c253fa-d100-40e7-8870-b195ceac0e00`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011873960495', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-2172::DEBUG::2012-12-09 12:52:35,723::task::588::TaskManager.Task::(_updateState) Task=`03c253fa-d100-40e7-8870-b195ceac0e00`::moving from state preparing -> state finished
>Thread-2172::DEBUG::2012-12-09 12:52:35,723::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2172::DEBUG::2012-12-09 12:52:35,723::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2172::DEBUG::2012-12-09 12:52:35,723::task::978::TaskManager.Task::(_decref) Task=`03c253fa-d100-40e7-8870-b195ceac0e00`::ref 0 aborting False
>Thread-2174::DEBUG::2012-12-09 12:52:35,857::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2174::DEBUG::2012-12-09 12:52:35,857::task::588::TaskManager.Task::(_updateState) Task=`9f1bf22e-c54c-4f17-beaa-72c000de84a2`::moving from state init -> state preparing
>Thread-2174::INFO::2012-12-09 12:52:35,858::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2174::INFO::2012-12-09 12:52:35,858::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2174::DEBUG::2012-12-09 12:52:35,858::task::1172::TaskManager.Task::(prepare) Task=`9f1bf22e-c54c-4f17-beaa-72c000de84a2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2174::DEBUG::2012-12-09 12:52:35,858::task::588::TaskManager.Task::(_updateState) Task=`9f1bf22e-c54c-4f17-beaa-72c000de84a2`::moving from state preparing -> state finished
>Thread-2174::DEBUG::2012-12-09 12:52:35,858::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2174::DEBUG::2012-12-09 12:52:35,859::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2174::DEBUG::2012-12-09 12:52:35,859::task::978::TaskManager.Task::(_decref) Task=`9f1bf22e-c54c-4f17-beaa-72c000de84a2`::ref 0 aborting False
>Thread-2175::DEBUG::2012-12-09 12:52:35,871::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2175::DEBUG::2012-12-09 12:52:35,871::task::588::TaskManager.Task::(_updateState) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::moving from state init -> state preparing
>Thread-2175::INFO::2012-12-09 12:52:35,872::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2175::DEBUG::2012-12-09 12:52:35,872::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5815f056-7253-4021-8e47-b8b65d16a5fb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2175::DEBUG::2012-12-09 12:52:35,872::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2175::DEBUG::2012-12-09 12:52:35,873::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2175::DEBUG::2012-12-09 12:52:35,873::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5815f056-7253-4021-8e47-b8b65d16a5fb`::Granted request
>Thread-2175::DEBUG::2012-12-09 12:52:35,873::task::817::TaskManager.Task::(resourceAcquired) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2175::DEBUG::2012-12-09 12:52:35,873::task::978::TaskManager.Task::(_decref) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::ref 1 aborting False
>Thread-2175::DEBUG::2012-12-09 12:52:35,874::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2175::DEBUG::2012-12-09 12:52:35,874::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2175::DEBUG::2012-12-09 12:52:35,974::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2175::WARNING::2012-12-09 12:52:35,975::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2175::DEBUG::2012-12-09 12:52:35,976::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2175::WARNING::2012-12-09 12:52:35,976::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2175::INFO::2012-12-09 12:52:35,976::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2175::DEBUG::2012-12-09 12:52:35,977::task::1172::TaskManager.Task::(prepare) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2175::DEBUG::2012-12-09 12:52:35,977::task::588::TaskManager.Task::(_updateState) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::moving from state preparing -> state finished
>Thread-2175::DEBUG::2012-12-09 12:52:35,977::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2175::DEBUG::2012-12-09 12:52:35,977::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2175::DEBUG::2012-12-09 12:52:35,977::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2175::DEBUG::2012-12-09 12:52:35,978::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2175::DEBUG::2012-12-09 12:52:35,978::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2175::DEBUG::2012-12-09 12:52:35,978::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2175::DEBUG::2012-12-09 12:52:35,978::task::978::TaskManager.Task::(_decref) Task=`577a4b5d-8016-44b5-a921-1423f8cad4fe`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:52:36,863::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:36,902::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016413 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:38,908::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:38,952::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019507 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:40,958::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:41,000::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170411 s, 60.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:52:41,341::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:41,353::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000375384 s, 10.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:42,485::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:42,486::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:42,586::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:42,587::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:42,587::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:42,588::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:43,007::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:43,049::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175449 s, 58.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:45,056::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:45,096::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163548 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:47,103::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:47,142::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164942 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:49,149::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:49,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165115 s, 62.0 MB/s\n'; <rc> = 0
>Thread-2180::DEBUG::2012-12-09 12:52:49,487::task::588::TaskManager.Task::(_updateState) Task=`1b056a79-9b39-4aff-9cb0-2855b286e73f`::moving from state init -> state preparing
>Thread-2180::INFO::2012-12-09 12:52:49,488::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2180::INFO::2012-12-09 12:52:49,488::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121419429779', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-2180::DEBUG::2012-12-09 12:52:49,489::task::1172::TaskManager.Task::(prepare) Task=`1b056a79-9b39-4aff-9cb0-2855b286e73f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0121419429779', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-2180::DEBUG::2012-12-09 12:52:49,489::task::588::TaskManager.Task::(_updateState) Task=`1b056a79-9b39-4aff-9cb0-2855b286e73f`::moving from state preparing -> state finished
>Thread-2180::DEBUG::2012-12-09 12:52:49,489::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2180::DEBUG::2012-12-09 12:52:49,490::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2180::DEBUG::2012-12-09 12:52:49,490::task::978::TaskManager.Task::(_decref) Task=`1b056a79-9b39-4aff-9cb0-2855b286e73f`::ref 0 aborting False
>Thread-2182::DEBUG::2012-12-09 12:52:49,908::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2182::DEBUG::2012-12-09 12:52:49,908::task::588::TaskManager.Task::(_updateState) Task=`b8465ed0-bce3-4c7f-9dc4-1d44648cd1df`::moving from state init -> state preparing
>Thread-2182::INFO::2012-12-09 12:52:49,909::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2182::INFO::2012-12-09 12:52:49,909::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2182::DEBUG::2012-12-09 12:52:49,909::task::1172::TaskManager.Task::(prepare) Task=`b8465ed0-bce3-4c7f-9dc4-1d44648cd1df`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2182::DEBUG::2012-12-09 12:52:49,909::task::588::TaskManager.Task::(_updateState) Task=`b8465ed0-bce3-4c7f-9dc4-1d44648cd1df`::moving from state preparing -> state finished
>Thread-2182::DEBUG::2012-12-09 12:52:49,910::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2182::DEBUG::2012-12-09 12:52:49,910::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2182::DEBUG::2012-12-09 12:52:49,910::task::978::TaskManager.Task::(_decref) Task=`b8465ed0-bce3-4c7f-9dc4-1d44648cd1df`::ref 0 aborting False
>Thread-2183::DEBUG::2012-12-09 12:52:49,921::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2183::DEBUG::2012-12-09 12:52:49,922::task::588::TaskManager.Task::(_updateState) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::moving from state init -> state preparing
>Thread-2183::INFO::2012-12-09 12:52:49,922::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2183::DEBUG::2012-12-09 12:52:49,923::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5e9d1143-a000-4992-aee1-f50c13cdd8a3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2183::DEBUG::2012-12-09 12:52:49,923::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2183::DEBUG::2012-12-09 12:52:49,923::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2183::DEBUG::2012-12-09 12:52:49,924::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5e9d1143-a000-4992-aee1-f50c13cdd8a3`::Granted request
>Thread-2183::DEBUG::2012-12-09 12:52:49,924::task::817::TaskManager.Task::(resourceAcquired) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2183::DEBUG::2012-12-09 12:52:49,925::task::978::TaskManager.Task::(_decref) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::ref 1 aborting False
>Thread-2183::DEBUG::2012-12-09 12:52:49,926::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2183::DEBUG::2012-12-09 12:52:49,927::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2183::DEBUG::2012-12-09 12:52:50,033::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2183::WARNING::2012-12-09 12:52:50,034::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2183::DEBUG::2012-12-09 12:52:50,035::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2183::WARNING::2012-12-09 12:52:50,035::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2183::INFO::2012-12-09 12:52:50,036::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2183::DEBUG::2012-12-09 12:52:50,036::task::1172::TaskManager.Task::(prepare) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2183::DEBUG::2012-12-09 12:52:50,037::task::588::TaskManager.Task::(_updateState) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::moving from state preparing -> state finished
>Thread-2183::DEBUG::2012-12-09 12:52:50,037::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2183::DEBUG::2012-12-09 12:52:50,037::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2183::DEBUG::2012-12-09 12:52:50,038::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2183::DEBUG::2012-12-09 12:52:50,038::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2183::DEBUG::2012-12-09 12:52:50,038::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2183::DEBUG::2012-12-09 12:52:50,038::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2183::DEBUG::2012-12-09 12:52:50,039::task::978::TaskManager.Task::(_decref) Task=`54eadfc8-0491-4dc3-a489-20badf459c36`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:52:51,199::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:51,243::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01653 s, 61.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:52:51,356::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:52:51,371::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000872319 s, 4.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:52:52,589::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:52:52,589::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:52:52,687::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:52:52,688::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:52:52,688::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:52:52,689::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:52:53,250::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:53,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020023 s, 51.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:55,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:55,339::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170119 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:57,346::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:57,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170157 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:52:59,393::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:52:59,434::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167258 s, 61.2 MB/s\n'; <rc> = 0
>Thread-2188::DEBUG::2012-12-09 12:53:00,418::task::588::TaskManager.Task::(_updateState) Task=`2c690589-b214-4c5b-8365-de5a95b554f5`::moving from state init -> state preparing
>Thread-2188::INFO::2012-12-09 12:53:00,418::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2188::INFO::2012-12-09 12:53:00,419::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164399147034', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2188::DEBUG::2012-12-09 12:53:00,419::task::1172::TaskManager.Task::(prepare) Task=`2c690589-b214-4c5b-8365-de5a95b554f5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0164399147034', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2188::DEBUG::2012-12-09 12:53:00,419::task::588::TaskManager.Task::(_updateState) Task=`2c690589-b214-4c5b-8365-de5a95b554f5`::moving from state preparing -> state finished
>Thread-2188::DEBUG::2012-12-09 12:53:00,419::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2188::DEBUG::2012-12-09 12:53:00,419::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2188::DEBUG::2012-12-09 12:53:00,419::task::978::TaskManager.Task::(_decref) Task=`2c690589-b214-4c5b-8365-de5a95b554f5`::ref 0 aborting False
>Thread-2190::DEBUG::2012-12-09 12:53:01,251::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2190::DEBUG::2012-12-09 12:53:01,252::task::588::TaskManager.Task::(_updateState) Task=`d4442c54-e831-4175-bb89-d31996239c37`::moving from state init -> state preparing
>Thread-2190::INFO::2012-12-09 12:53:01,252::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2190::INFO::2012-12-09 12:53:01,252::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2190::DEBUG::2012-12-09 12:53:01,253::task::1172::TaskManager.Task::(prepare) Task=`d4442c54-e831-4175-bb89-d31996239c37`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2190::DEBUG::2012-12-09 12:53:01,253::task::588::TaskManager.Task::(_updateState) Task=`d4442c54-e831-4175-bb89-d31996239c37`::moving from state preparing -> state finished
>Thread-2190::DEBUG::2012-12-09 12:53:01,253::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2190::DEBUG::2012-12-09 12:53:01,253::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2190::DEBUG::2012-12-09 12:53:01,253::task::978::TaskManager.Task::(_decref) Task=`d4442c54-e831-4175-bb89-d31996239c37`::ref 0 aborting False
>Thread-2191::DEBUG::2012-12-09 12:53:01,267::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2191::DEBUG::2012-12-09 12:53:01,267::task::588::TaskManager.Task::(_updateState) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::moving from state init -> state preparing
>Thread-2191::INFO::2012-12-09 12:53:01,267::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2191::DEBUG::2012-12-09 12:53:01,268::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c078450-7035-49b6-8ff3-bc3e00e621f4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2191::DEBUG::2012-12-09 12:53:01,268::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2191::DEBUG::2012-12-09 12:53:01,268::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2191::DEBUG::2012-12-09 12:53:01,268::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c078450-7035-49b6-8ff3-bc3e00e621f4`::Granted request
>Thread-2191::DEBUG::2012-12-09 12:53:01,268::task::817::TaskManager.Task::(resourceAcquired) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2191::DEBUG::2012-12-09 12:53:01,269::task::978::TaskManager.Task::(_decref) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::ref 1 aborting False
>Thread-2191::DEBUG::2012-12-09 12:53:01,269::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2191::DEBUG::2012-12-09 12:53:01,270::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2191::DEBUG::2012-12-09 12:53:01,369::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2191::WARNING::2012-12-09 12:53:01,370::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2191::DEBUG::2012-12-09 12:53:01,371::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2191::WARNING::2012-12-09 12:53:01,371::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2191::INFO::2012-12-09 12:53:01,371::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2191::DEBUG::2012-12-09 12:53:01,372::task::1172::TaskManager.Task::(prepare) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2191::DEBUG::2012-12-09 12:53:01,372::task::588::TaskManager.Task::(_updateState) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::moving from state preparing -> state finished
>Thread-2191::DEBUG::2012-12-09 12:53:01,372::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2191::DEBUG::2012-12-09 12:53:01,372::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2191::DEBUG::2012-12-09 12:53:01,372::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2191::DEBUG::2012-12-09 12:53:01,373::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2191::DEBUG::2012-12-09 12:53:01,373::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2191::DEBUG::2012-12-09 12:53:01,373::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2191::DEBUG::2012-12-09 12:53:01,373::task::978::TaskManager.Task::(_decref) Task=`4d06064e-d812-4922-9e3a-c7f69715ce71`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:53:01,377::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:01,384::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000369767 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:01,441::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:01,480::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164868 s, 62.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:02,690::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:02,690::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:02,788::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:02,789::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:02,789::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:02,790::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:53:03,487::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:03,529::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193988 s, 52.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:05,536::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:05,576::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178159 s, 57.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:07,583::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:07,624::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171117 s, 59.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:09,631::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:09,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016441 s, 62.3 MB/s\n'; <rc> = 0
>Thread-2196::DEBUG::2012-12-09 12:53:11,213::task::588::TaskManager.Task::(_updateState) Task=`0e86649d-87cd-4d73-a293-b3a59fe03912`::moving from state init -> state preparing
>Thread-2196::INFO::2012-12-09 12:53:11,214::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2196::INFO::2012-12-09 12:53:11,214::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00758600234985', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-2196::DEBUG::2012-12-09 12:53:11,214::task::1172::TaskManager.Task::(prepare) Task=`0e86649d-87cd-4d73-a293-b3a59fe03912`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00758600234985', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-2196::DEBUG::2012-12-09 12:53:11,215::task::588::TaskManager.Task::(_updateState) Task=`0e86649d-87cd-4d73-a293-b3a59fe03912`::moving from state preparing -> state finished
>Thread-2196::DEBUG::2012-12-09 12:53:11,215::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2196::DEBUG::2012-12-09 12:53:11,215::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2196::DEBUG::2012-12-09 12:53:11,215::task::978::TaskManager.Task::(_decref) Task=`0e86649d-87cd-4d73-a293-b3a59fe03912`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:53:11,386::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:11,394::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000395132 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:11,677::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:11,716::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158333 s, 64.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:12,790::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:12,791::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:12,886::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:12,887::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:12,888::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:12,888::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2198::DEBUG::2012-12-09 12:53:13,208::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2198::DEBUG::2012-12-09 12:53:13,209::task::588::TaskManager.Task::(_updateState) Task=`b471d97c-1212-49f9-8b49-86f90ec073dd`::moving from state init -> state preparing
>Thread-2198::INFO::2012-12-09 12:53:13,209::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2198::INFO::2012-12-09 12:53:13,209::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2198::DEBUG::2012-12-09 12:53:13,210::task::1172::TaskManager.Task::(prepare) Task=`b471d97c-1212-49f9-8b49-86f90ec073dd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2198::DEBUG::2012-12-09 12:53:13,210::task::588::TaskManager.Task::(_updateState) Task=`b471d97c-1212-49f9-8b49-86f90ec073dd`::moving from state preparing -> state finished
>Thread-2198::DEBUG::2012-12-09 12:53:13,210::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2198::DEBUG::2012-12-09 12:53:13,210::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2198::DEBUG::2012-12-09 12:53:13,210::task::978::TaskManager.Task::(_decref) Task=`b471d97c-1212-49f9-8b49-86f90ec073dd`::ref 0 aborting False
>Thread-2199::DEBUG::2012-12-09 12:53:13,225::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2199::DEBUG::2012-12-09 12:53:13,225::task::588::TaskManager.Task::(_updateState) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::moving from state init -> state preparing
>Thread-2199::INFO::2012-12-09 12:53:13,226::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2199::DEBUG::2012-12-09 12:53:13,226::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`eb758d06-f361-49ec-bcfc-f7e650ea8414`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2199::DEBUG::2012-12-09 12:53:13,226::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2199::DEBUG::2012-12-09 12:53:13,226::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2199::DEBUG::2012-12-09 12:53:13,227::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`eb758d06-f361-49ec-bcfc-f7e650ea8414`::Granted request
>Thread-2199::DEBUG::2012-12-09 12:53:13,227::task::817::TaskManager.Task::(resourceAcquired) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2199::DEBUG::2012-12-09 12:53:13,227::task::978::TaskManager.Task::(_decref) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::ref 1 aborting False
>Thread-2199::DEBUG::2012-12-09 12:53:13,228::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2199::DEBUG::2012-12-09 12:53:13,228::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2199::DEBUG::2012-12-09 12:53:13,327::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2199::WARNING::2012-12-09 12:53:13,329::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2199::DEBUG::2012-12-09 12:53:13,329::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2199::WARNING::2012-12-09 12:53:13,330::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2199::INFO::2012-12-09 12:53:13,330::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2199::DEBUG::2012-12-09 12:53:13,331::task::1172::TaskManager.Task::(prepare) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2199::DEBUG::2012-12-09 12:53:13,331::task::588::TaskManager.Task::(_updateState) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::moving from state preparing -> state finished
>Thread-2199::DEBUG::2012-12-09 12:53:13,331::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2199::DEBUG::2012-12-09 12:53:13,332::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2199::DEBUG::2012-12-09 12:53:13,332::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2199::DEBUG::2012-12-09 12:53:13,332::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2199::DEBUG::2012-12-09 12:53:13,333::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2199::DEBUG::2012-12-09 12:53:13,333::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2199::DEBUG::2012-12-09 12:53:13,333::task::978::TaskManager.Task::(_decref) Task=`297197ba-ccd1-4b74-84e9-af1aec77e9c6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:53:13,722::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:13,761::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167006 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:15,768::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:15,807::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158529 s, 64.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:17,813::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:17,862::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227847 s, 44.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:19,868::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:19,908::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166961 s, 61.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:53:21,397::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:21,409::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000269422 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:21,915::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:21,955::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163122 s, 62.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:22,889::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:22,890::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:23,001::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:23,002::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:23,003::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:23,003::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:53:23,962::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:24,006::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192108 s, 53.3 MB/s\n'; <rc> = 0
>Thread-2204::DEBUG::2012-12-09 12:53:24,883::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2204::DEBUG::2012-12-09 12:53:24,884::task::588::TaskManager.Task::(_updateState) Task=`b9a26c99-82ef-4742-b60b-4469da0d30de`::moving from state init -> state preparing
>Thread-2204::INFO::2012-12-09 12:53:24,884::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2204::INFO::2012-12-09 12:53:24,884::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2204::DEBUG::2012-12-09 12:53:24,884::task::1172::TaskManager.Task::(prepare) Task=`b9a26c99-82ef-4742-b60b-4469da0d30de`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2204::DEBUG::2012-12-09 12:53:24,884::task::588::TaskManager.Task::(_updateState) Task=`b9a26c99-82ef-4742-b60b-4469da0d30de`::moving from state preparing -> state finished
>Thread-2204::DEBUG::2012-12-09 12:53:24,885::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2204::DEBUG::2012-12-09 12:53:24,885::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2204::DEBUG::2012-12-09 12:53:24,885::task::978::TaskManager.Task::(_decref) Task=`b9a26c99-82ef-4742-b60b-4469da0d30de`::ref 0 aborting False
>Thread-2205::DEBUG::2012-12-09 12:53:24,898::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2205::DEBUG::2012-12-09 12:53:24,898::task::588::TaskManager.Task::(_updateState) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::moving from state init -> state preparing
>Thread-2205::INFO::2012-12-09 12:53:24,899::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2205::DEBUG::2012-12-09 12:53:24,899::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`26d967d9-db59-4cfe-9a6d-6173a84054c9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2205::DEBUG::2012-12-09 12:53:24,899::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2205::DEBUG::2012-12-09 12:53:24,899::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2205::DEBUG::2012-12-09 12:53:24,900::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`26d967d9-db59-4cfe-9a6d-6173a84054c9`::Granted request
>Thread-2205::DEBUG::2012-12-09 12:53:24,900::task::817::TaskManager.Task::(resourceAcquired) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2205::DEBUG::2012-12-09 12:53:24,900::task::978::TaskManager.Task::(_decref) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::ref 1 aborting False
>Thread-2205::DEBUG::2012-12-09 12:53:24,901::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2205::DEBUG::2012-12-09 12:53:24,901::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2205::DEBUG::2012-12-09 12:53:25,004::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2205::WARNING::2012-12-09 12:53:25,005::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2205::DEBUG::2012-12-09 12:53:25,006::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2205::WARNING::2012-12-09 12:53:25,006::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2205::INFO::2012-12-09 12:53:25,006::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2205::DEBUG::2012-12-09 12:53:25,007::task::1172::TaskManager.Task::(prepare) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2205::DEBUG::2012-12-09 12:53:25,007::task::588::TaskManager.Task::(_updateState) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::moving from state preparing -> state finished
>Thread-2205::DEBUG::2012-12-09 12:53:25,007::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2205::DEBUG::2012-12-09 12:53:25,007::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2205::DEBUG::2012-12-09 12:53:25,008::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2205::DEBUG::2012-12-09 12:53:25,008::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2205::DEBUG::2012-12-09 12:53:25,008::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2205::DEBUG::2012-12-09 12:53:25,008::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2205::DEBUG::2012-12-09 12:53:25,008::task::978::TaskManager.Task::(_decref) Task=`76b07a84-c782-4cd9-a774-660cdea40a83`::ref 0 aborting False
>Thread-2206::DEBUG::2012-12-09 12:53:25,204::task::588::TaskManager.Task::(_updateState) Task=`d61f8101-3802-4f00-81c0-4d899d57239b`::moving from state init -> state preparing
>Thread-2206::INFO::2012-12-09 12:53:25,204::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2206::INFO::2012-12-09 12:53:25,205::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128149986267', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-2206::DEBUG::2012-12-09 12:53:25,205::task::1172::TaskManager.Task::(prepare) Task=`d61f8101-3802-4f00-81c0-4d899d57239b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128149986267', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.2', 'code': 358, 'valid': False}}
>Thread-2206::DEBUG::2012-12-09 12:53:25,206::task::588::TaskManager.Task::(_updateState) Task=`d61f8101-3802-4f00-81c0-4d899d57239b`::moving from state preparing -> state finished
>Thread-2206::DEBUG::2012-12-09 12:53:25,206::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2206::DEBUG::2012-12-09 12:53:25,207::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2206::DEBUG::2012-12-09 12:53:25,207::task::978::TaskManager.Task::(_decref) Task=`d61f8101-3802-4f00-81c0-4d899d57239b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:53:26,012::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:26,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156396 s, 65.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:28,057::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:28,096::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165742 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:30,103::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:30,142::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161836 s, 63.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:53:31,412::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:31,420::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000388736 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:32,148::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:32,190::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177331 s, 57.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:33,004::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:33,005::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:33,105::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:33,106::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:33,106::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:33,107::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:53:34,197::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:34,237::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169405 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:36,244::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:36,295::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02824 s, 36.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:38,302::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:38,348::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221954 s, 46.1 MB/s\n'; <rc> = 0
>Thread-2212::DEBUG::2012-12-09 12:53:38,996::task::588::TaskManager.Task::(_updateState) Task=`51673721-fde7-4b68-9260-d69c7384685a`::moving from state init -> state preparing
>Thread-2212::INFO::2012-12-09 12:53:38,997::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2212::INFO::2012-12-09 12:53:38,997::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00819087028503', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-2212::DEBUG::2012-12-09 12:53:38,997::task::1172::TaskManager.Task::(prepare) Task=`51673721-fde7-4b68-9260-d69c7384685a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00819087028503', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-2212::DEBUG::2012-12-09 12:53:38,997::task::588::TaskManager.Task::(_updateState) Task=`51673721-fde7-4b68-9260-d69c7384685a`::moving from state preparing -> state finished
>Thread-2212::DEBUG::2012-12-09 12:53:38,998::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2212::DEBUG::2012-12-09 12:53:38,998::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2212::DEBUG::2012-12-09 12:53:38,998::task::978::TaskManager.Task::(_decref) Task=`51673721-fde7-4b68-9260-d69c7384685a`::ref 0 aborting False
>Thread-2214::DEBUG::2012-12-09 12:53:39,097::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2214::DEBUG::2012-12-09 12:53:39,097::task::588::TaskManager.Task::(_updateState) Task=`aea3403e-45a2-4efa-8d0e-e17ef9a75762`::moving from state init -> state preparing
>Thread-2214::INFO::2012-12-09 12:53:39,098::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2214::INFO::2012-12-09 12:53:39,098::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2214::DEBUG::2012-12-09 12:53:39,098::task::1172::TaskManager.Task::(prepare) Task=`aea3403e-45a2-4efa-8d0e-e17ef9a75762`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2214::DEBUG::2012-12-09 12:53:39,098::task::588::TaskManager.Task::(_updateState) Task=`aea3403e-45a2-4efa-8d0e-e17ef9a75762`::moving from state preparing -> state finished
>Thread-2214::DEBUG::2012-12-09 12:53:39,099::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2214::DEBUG::2012-12-09 12:53:39,099::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2214::DEBUG::2012-12-09 12:53:39,099::task::978::TaskManager.Task::(_decref) Task=`aea3403e-45a2-4efa-8d0e-e17ef9a75762`::ref 0 aborting False
>Thread-2215::DEBUG::2012-12-09 12:53:39,111::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2215::DEBUG::2012-12-09 12:53:39,111::task::588::TaskManager.Task::(_updateState) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::moving from state init -> state preparing
>Thread-2215::INFO::2012-12-09 12:53:39,111::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2215::DEBUG::2012-12-09 12:53:39,112::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`089c2392-eb22-4868-9ab0-ac4999423144`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2215::DEBUG::2012-12-09 12:53:39,112::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2215::DEBUG::2012-12-09 12:53:39,112::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2215::DEBUG::2012-12-09 12:53:39,112::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`089c2392-eb22-4868-9ab0-ac4999423144`::Granted request
>Thread-2215::DEBUG::2012-12-09 12:53:39,113::task::817::TaskManager.Task::(resourceAcquired) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2215::DEBUG::2012-12-09 12:53:39,113::task::978::TaskManager.Task::(_decref) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::ref 1 aborting False
>Thread-2215::DEBUG::2012-12-09 12:53:39,114::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2215::DEBUG::2012-12-09 12:53:39,114::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2215::DEBUG::2012-12-09 12:53:39,211::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2215::WARNING::2012-12-09 12:53:39,212::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2215::DEBUG::2012-12-09 12:53:39,213::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2215::WARNING::2012-12-09 12:53:39,213::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2215::INFO::2012-12-09 12:53:39,214::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2215::DEBUG::2012-12-09 12:53:39,214::task::1172::TaskManager.Task::(prepare) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2215::DEBUG::2012-12-09 12:53:39,214::task::588::TaskManager.Task::(_updateState) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::moving from state preparing -> state finished
>Thread-2215::DEBUG::2012-12-09 12:53:39,214::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2215::DEBUG::2012-12-09 12:53:39,215::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2215::DEBUG::2012-12-09 12:53:39,215::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2215::DEBUG::2012-12-09 12:53:39,215::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2215::DEBUG::2012-12-09 12:53:39,215::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2215::DEBUG::2012-12-09 12:53:39,216::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2215::DEBUG::2012-12-09 12:53:39,216::task::978::TaskManager.Task::(_decref) Task=`8b1b26db-18b9-411a-a4b8-d1c8685b014a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:53:40,355::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:40,393::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160035 s, 64.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:53:41,423::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:41,431::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000412249 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:42,400::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:42,439::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164739 s, 62.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:43,108::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:43,108::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:43,219::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:43,220::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:43,220::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:43,221::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:53:44,446::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:44,484::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158922 s, 64.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:46,491::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:46,532::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168674 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:48,538::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:48,579::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169877 s, 60.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:50,586::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:50,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173512 s, 59.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:53:51,433::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:53:51,445::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000398239 s, 10.3 MB/s\n'; <rc> = 0
>Thread-2220::DEBUG::2012-12-09 12:53:52,288::task::588::TaskManager.Task::(_updateState) Task=`a78ed578-3dcd-49c3-b951-3c674618d6bc`::moving from state init -> state preparing
>Thread-2220::INFO::2012-12-09 12:53:52,288::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2220::INFO::2012-12-09 12:53:52,289::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120661258698', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-2220::DEBUG::2012-12-09 12:53:52,289::task::1172::TaskManager.Task::(prepare) Task=`a78ed578-3dcd-49c3-b951-3c674618d6bc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120661258698', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-2220::DEBUG::2012-12-09 12:53:52,290::task::588::TaskManager.Task::(_updateState) Task=`a78ed578-3dcd-49c3-b951-3c674618d6bc`::moving from state preparing -> state finished
>Thread-2220::DEBUG::2012-12-09 12:53:52,290::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2220::DEBUG::2012-12-09 12:53:52,290::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2220::DEBUG::2012-12-09 12:53:52,290::task::978::TaskManager.Task::(_decref) Task=`a78ed578-3dcd-49c3-b951-3c674618d6bc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:53:52,635::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:52,672::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161223 s, 63.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:53:53,222::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:53:53,223::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:53:53,319::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:53:53,320::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:53:53,321::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:53:53,321::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2222::DEBUG::2012-12-09 12:53:53,363::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2222::DEBUG::2012-12-09 12:53:53,363::task::588::TaskManager.Task::(_updateState) Task=`63bbc7df-a46d-4b15-a375-8968cde0be44`::moving from state init -> state preparing
>Thread-2222::INFO::2012-12-09 12:53:53,363::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2222::INFO::2012-12-09 12:53:53,364::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2222::DEBUG::2012-12-09 12:53:53,364::task::1172::TaskManager.Task::(prepare) Task=`63bbc7df-a46d-4b15-a375-8968cde0be44`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2222::DEBUG::2012-12-09 12:53:53,364::task::588::TaskManager.Task::(_updateState) Task=`63bbc7df-a46d-4b15-a375-8968cde0be44`::moving from state preparing -> state finished
>Thread-2222::DEBUG::2012-12-09 12:53:53,364::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2222::DEBUG::2012-12-09 12:53:53,364::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2222::DEBUG::2012-12-09 12:53:53,365::task::978::TaskManager.Task::(_decref) Task=`63bbc7df-a46d-4b15-a375-8968cde0be44`::ref 0 aborting False
>Thread-2223::DEBUG::2012-12-09 12:53:53,377::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2223::DEBUG::2012-12-09 12:53:53,377::task::588::TaskManager.Task::(_updateState) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::moving from state init -> state preparing
>Thread-2223::INFO::2012-12-09 12:53:53,378::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2223::DEBUG::2012-12-09 12:53:53,378::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`21145083-7c19-4b80-93f3-b8d6caf2549f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2223::DEBUG::2012-12-09 12:53:53,378::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2223::DEBUG::2012-12-09 12:53:53,378::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2223::DEBUG::2012-12-09 12:53:53,379::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`21145083-7c19-4b80-93f3-b8d6caf2549f`::Granted request
>Thread-2223::DEBUG::2012-12-09 12:53:53,379::task::817::TaskManager.Task::(resourceAcquired) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2223::DEBUG::2012-12-09 12:53:53,379::task::978::TaskManager.Task::(_decref) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::ref 1 aborting False
>Thread-2223::DEBUG::2012-12-09 12:53:53,380::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2223::DEBUG::2012-12-09 12:53:53,380::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2223::DEBUG::2012-12-09 12:53:53,481::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2223::WARNING::2012-12-09 12:53:53,482::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2223::DEBUG::2012-12-09 12:53:53,482::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2223::WARNING::2012-12-09 12:53:53,483::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2223::INFO::2012-12-09 12:53:53,483::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2223::DEBUG::2012-12-09 12:53:53,483::task::1172::TaskManager.Task::(prepare) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2223::DEBUG::2012-12-09 12:53:53,484::task::588::TaskManager.Task::(_updateState) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::moving from state preparing -> state finished
>Thread-2223::DEBUG::2012-12-09 12:53:53,484::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2223::DEBUG::2012-12-09 12:53:53,484::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2223::DEBUG::2012-12-09 12:53:53,484::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2223::DEBUG::2012-12-09 12:53:53,484::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2223::DEBUG::2012-12-09 12:53:53,485::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2223::DEBUG::2012-12-09 12:53:53,485::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2223::DEBUG::2012-12-09 12:53:53,485::task::978::TaskManager.Task::(_decref) Task=`f560af2e-c667-4e75-8658-a774c4a1a422`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:53:54,679::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:54,717::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016277 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:56,724::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:56,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215105 s, 47.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:53:58,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:53:58,820::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203049 s, 50.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:00,826::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:00,865::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168638 s, 60.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:01,448::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:01,459::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000354109 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:02,872::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:02,923::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259157 s, 39.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:03,322::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:03,323::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2228::DEBUG::2012-12-09 12:54:03,357::task::588::TaskManager.Task::(_updateState) Task=`6ec31d85-8550-4d14-ada6-1091b0efe41d`::moving from state init -> state preparing
>Thread-2228::INFO::2012-12-09 12:54:03,358::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2228::INFO::2012-12-09 12:54:03,358::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0110890865326', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-2228::DEBUG::2012-12-09 12:54:03,359::task::1172::TaskManager.Task::(prepare) Task=`6ec31d85-8550-4d14-ada6-1091b0efe41d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0110890865326', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-2228::DEBUG::2012-12-09 12:54:03,359::task::588::TaskManager.Task::(_updateState) Task=`6ec31d85-8550-4d14-ada6-1091b0efe41d`::moving from state preparing -> state finished
>Thread-2228::DEBUG::2012-12-09 12:54:03,359::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2228::DEBUG::2012-12-09 12:54:03,360::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2228::DEBUG::2012-12-09 12:54:03,360::task::978::TaskManager.Task::(_decref) Task=`6ec31d85-8550-4d14-ada6-1091b0efe41d`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:54:03,425::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:03,426::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:03,426::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:03,427::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2230::DEBUG::2012-12-09 12:54:04,808::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2230::DEBUG::2012-12-09 12:54:04,809::task::588::TaskManager.Task::(_updateState) Task=`3c3f2779-937f-40b4-a17f-aa3fc2e346ee`::moving from state init -> state preparing
>Thread-2230::INFO::2012-12-09 12:54:04,809::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2230::INFO::2012-12-09 12:54:04,809::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2230::DEBUG::2012-12-09 12:54:04,809::task::1172::TaskManager.Task::(prepare) Task=`3c3f2779-937f-40b4-a17f-aa3fc2e346ee`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2230::DEBUG::2012-12-09 12:54:04,809::task::588::TaskManager.Task::(_updateState) Task=`3c3f2779-937f-40b4-a17f-aa3fc2e346ee`::moving from state preparing -> state finished
>Thread-2230::DEBUG::2012-12-09 12:54:04,810::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2230::DEBUG::2012-12-09 12:54:04,810::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2230::DEBUG::2012-12-09 12:54:04,810::task::978::TaskManager.Task::(_decref) Task=`3c3f2779-937f-40b4-a17f-aa3fc2e346ee`::ref 0 aborting False
>Thread-2231::DEBUG::2012-12-09 12:54:04,823::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2231::DEBUG::2012-12-09 12:54:04,824::task::588::TaskManager.Task::(_updateState) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::moving from state init -> state preparing
>Thread-2231::INFO::2012-12-09 12:54:04,824::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2231::DEBUG::2012-12-09 12:54:04,824::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bbc997c7-dd48-4574-9fe1-2949de02f509`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2231::DEBUG::2012-12-09 12:54:04,824::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2231::DEBUG::2012-12-09 12:54:04,825::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2231::DEBUG::2012-12-09 12:54:04,825::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bbc997c7-dd48-4574-9fe1-2949de02f509`::Granted request
>Thread-2231::DEBUG::2012-12-09 12:54:04,825::task::817::TaskManager.Task::(resourceAcquired) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2231::DEBUG::2012-12-09 12:54:04,826::task::978::TaskManager.Task::(_decref) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::ref 1 aborting False
>Thread-2231::DEBUG::2012-12-09 12:54:04,826::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2231::DEBUG::2012-12-09 12:54:04,827::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2231::DEBUG::2012-12-09 12:54:04,923::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2231::WARNING::2012-12-09 12:54:04,924::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2231::DEBUG::2012-12-09 12:54:04,924::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2231::WARNING::2012-12-09 12:54:04,925::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2231::INFO::2012-12-09 12:54:04,925::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2231::DEBUG::2012-12-09 12:54:04,925::task::1172::TaskManager.Task::(prepare) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2231::DEBUG::2012-12-09 12:54:04,926::task::588::TaskManager.Task::(_updateState) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::moving from state preparing -> state finished
>Thread-2231::DEBUG::2012-12-09 12:54:04,926::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2231::DEBUG::2012-12-09 12:54:04,926::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2231::DEBUG::2012-12-09 12:54:04,926::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2231::DEBUG::2012-12-09 12:54:04,926::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2231::DEBUG::2012-12-09 12:54:04,927::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2231::DEBUG::2012-12-09 12:54:04,927::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2231::DEBUG::2012-12-09 12:54:04,927::task::978::TaskManager.Task::(_decref) Task=`7ca22171-bb54-43ed-8cf6-f3bf7bab6859`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:54:04,929::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:04,966::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160257 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:06,973::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:07,039::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0287766 s, 35.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:09,046::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:09,088::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169272 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:11,095::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:11,135::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161431 s, 63.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:11,461::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:54:11,462::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:54:11,462::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:11,472::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000389036 s, 5.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:11,473::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000389036 s, 5.3 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:54:11,473::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:54:11,473::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:54:11,473::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:54:11,473::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,474::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,474::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,474::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,474::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,475::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:11,570::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:11,571::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:54:11,572::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:54:11,572::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:54:11,572::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:54:11,573::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:54:11,573::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:11,668::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:11,669::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:11,680::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000380999 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:13,141::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:13,181::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170218 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:13,428::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:13,428::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:54:13,534::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:13,535::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:13,536::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:13,536::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:54:15,188::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:15,231::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019467 s, 52.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:17,238::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:17,280::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169474 s, 60.4 MB/s\n'; <rc> = 0
>Thread-2236::DEBUG::2012-12-09 12:54:17,524::task::588::TaskManager.Task::(_updateState) Task=`c549ae5c-c760-4436-b6e1-ac8474311caf`::moving from state init -> state preparing
>Thread-2236::INFO::2012-12-09 12:54:17,525::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2236::INFO::2012-12-09 12:54:17,525::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122919082642', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-2236::DEBUG::2012-12-09 12:54:17,526::task::1172::TaskManager.Task::(prepare) Task=`c549ae5c-c760-4436-b6e1-ac8474311caf`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122919082642', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.0', 'code': 358, 'valid': False}}
>Thread-2236::DEBUG::2012-12-09 12:54:17,526::task::588::TaskManager.Task::(_updateState) Task=`c549ae5c-c760-4436-b6e1-ac8474311caf`::moving from state preparing -> state finished
>Thread-2236::DEBUG::2012-12-09 12:54:17,526::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2236::DEBUG::2012-12-09 12:54:17,526::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2236::DEBUG::2012-12-09 12:54:17,527::task::978::TaskManager.Task::(_decref) Task=`c549ae5c-c760-4436-b6e1-ac8474311caf`::ref 0 aborting False
>Thread-2238::DEBUG::2012-12-09 12:54:17,888::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2238::DEBUG::2012-12-09 12:54:17,889::task::588::TaskManager.Task::(_updateState) Task=`c7fc1ac7-6bc8-4860-9b92-f20e347a21ce`::moving from state init -> state preparing
>Thread-2238::INFO::2012-12-09 12:54:17,889::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2238::INFO::2012-12-09 12:54:17,889::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2238::DEBUG::2012-12-09 12:54:17,890::task::1172::TaskManager.Task::(prepare) Task=`c7fc1ac7-6bc8-4860-9b92-f20e347a21ce`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2238::DEBUG::2012-12-09 12:54:17,890::task::588::TaskManager.Task::(_updateState) Task=`c7fc1ac7-6bc8-4860-9b92-f20e347a21ce`::moving from state preparing -> state finished
>Thread-2238::DEBUG::2012-12-09 12:54:17,890::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2238::DEBUG::2012-12-09 12:54:17,891::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2238::DEBUG::2012-12-09 12:54:17,891::task::978::TaskManager.Task::(_decref) Task=`c7fc1ac7-6bc8-4860-9b92-f20e347a21ce`::ref 0 aborting False
>Thread-2239::DEBUG::2012-12-09 12:54:17,905::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2239::DEBUG::2012-12-09 12:54:17,905::task::588::TaskManager.Task::(_updateState) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::moving from state init -> state preparing
>Thread-2239::INFO::2012-12-09 12:54:17,905::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2239::DEBUG::2012-12-09 12:54:17,906::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fb873ed8-5d1b-464b-8a89-fe6fd25214c4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2239::DEBUG::2012-12-09 12:54:17,906::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2239::DEBUG::2012-12-09 12:54:17,906::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2239::DEBUG::2012-12-09 12:54:17,906::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fb873ed8-5d1b-464b-8a89-fe6fd25214c4`::Granted request
>Thread-2239::DEBUG::2012-12-09 12:54:17,907::task::817::TaskManager.Task::(resourceAcquired) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2239::DEBUG::2012-12-09 12:54:17,907::task::978::TaskManager.Task::(_decref) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::ref 1 aborting False
>Thread-2239::DEBUG::2012-12-09 12:54:17,908::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2239::DEBUG::2012-12-09 12:54:17,908::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2239::DEBUG::2012-12-09 12:54:18,006::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2239::WARNING::2012-12-09 12:54:18,007::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2239::DEBUG::2012-12-09 12:54:18,007::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2239::WARNING::2012-12-09 12:54:18,008::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2239::INFO::2012-12-09 12:54:18,009::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2239::DEBUG::2012-12-09 12:54:18,009::task::1172::TaskManager.Task::(prepare) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2239::DEBUG::2012-12-09 12:54:18,009::task::588::TaskManager.Task::(_updateState) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::moving from state preparing -> state finished
>Thread-2239::DEBUG::2012-12-09 12:54:18,010::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2239::DEBUG::2012-12-09 12:54:18,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2239::DEBUG::2012-12-09 12:54:18,010::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2239::DEBUG::2012-12-09 12:54:18,010::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2239::DEBUG::2012-12-09 12:54:18,011::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2239::DEBUG::2012-12-09 12:54:18,011::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2239::DEBUG::2012-12-09 12:54:18,011::task::978::TaskManager.Task::(_decref) Task=`5827f5df-3490-40f3-9ae5-4055de9a2556`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:54:19,287::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:19,331::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016276 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:21,337::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:21,378::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183181 s, 55.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:21,684::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:21,695::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000355047 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:23,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:23,424::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155349 s, 65.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:23,537::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:23,538::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:54:23,634::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:23,635::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:23,635::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:23,636::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:54:25,431::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:25,472::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163216 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:27,479::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:27,528::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241145 s, 42.5 MB/s\n'; <rc> = 0
>Thread-2244::DEBUG::2012-12-09 12:54:28,114::task::588::TaskManager.Task::(_updateState) Task=`a9def7ae-d8d2-4382-9e99-465270b9a751`::moving from state init -> state preparing
>Thread-2244::INFO::2012-12-09 12:54:28,114::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2244::INFO::2012-12-09 12:54:28,115::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112969875336', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-2244::DEBUG::2012-12-09 12:54:28,115::task::1172::TaskManager.Task::(prepare) Task=`a9def7ae-d8d2-4382-9e99-465270b9a751`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0112969875336', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.5', 'code': 358, 'valid': False}}
>Thread-2244::DEBUG::2012-12-09 12:54:28,115::task::588::TaskManager.Task::(_updateState) Task=`a9def7ae-d8d2-4382-9e99-465270b9a751`::moving from state preparing -> state finished
>Thread-2244::DEBUG::2012-12-09 12:54:28,115::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2244::DEBUG::2012-12-09 12:54:28,115::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2244::DEBUG::2012-12-09 12:54:28,116::task::978::TaskManager.Task::(_decref) Task=`a9def7ae-d8d2-4382-9e99-465270b9a751`::ref 0 aborting False
>Thread-2246::DEBUG::2012-12-09 12:54:28,834::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2246::DEBUG::2012-12-09 12:54:28,835::task::588::TaskManager.Task::(_updateState) Task=`a9023bb4-6549-4634-b8de-a492eed964f2`::moving from state init -> state preparing
>Thread-2246::INFO::2012-12-09 12:54:28,835::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2246::INFO::2012-12-09 12:54:28,836::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2246::DEBUG::2012-12-09 12:54:28,836::task::1172::TaskManager.Task::(prepare) Task=`a9023bb4-6549-4634-b8de-a492eed964f2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2246::DEBUG::2012-12-09 12:54:28,836::task::588::TaskManager.Task::(_updateState) Task=`a9023bb4-6549-4634-b8de-a492eed964f2`::moving from state preparing -> state finished
>Thread-2246::DEBUG::2012-12-09 12:54:28,836::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2246::DEBUG::2012-12-09 12:54:28,837::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2246::DEBUG::2012-12-09 12:54:28,837::task::978::TaskManager.Task::(_decref) Task=`a9023bb4-6549-4634-b8de-a492eed964f2`::ref 0 aborting False
>Thread-2247::DEBUG::2012-12-09 12:54:28,850::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2247::DEBUG::2012-12-09 12:54:28,851::task::588::TaskManager.Task::(_updateState) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::moving from state init -> state preparing
>Thread-2247::INFO::2012-12-09 12:54:28,851::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2247::DEBUG::2012-12-09 12:54:28,852::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`03c03931-bcdb-4276-9237-4ae4de940915`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2247::DEBUG::2012-12-09 12:54:28,852::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2247::DEBUG::2012-12-09 12:54:28,852::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2247::DEBUG::2012-12-09 12:54:28,852::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`03c03931-bcdb-4276-9237-4ae4de940915`::Granted request
>Thread-2247::DEBUG::2012-12-09 12:54:28,852::task::817::TaskManager.Task::(resourceAcquired) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2247::DEBUG::2012-12-09 12:54:28,853::task::978::TaskManager.Task::(_decref) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::ref 1 aborting False
>Thread-2247::DEBUG::2012-12-09 12:54:28,853::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2247::DEBUG::2012-12-09 12:54:28,854::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2247::DEBUG::2012-12-09 12:54:28,956::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2247::WARNING::2012-12-09 12:54:28,957::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2247::DEBUG::2012-12-09 12:54:28,957::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2247::WARNING::2012-12-09 12:54:28,958::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2247::INFO::2012-12-09 12:54:28,958::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2247::DEBUG::2012-12-09 12:54:28,958::task::1172::TaskManager.Task::(prepare) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2247::DEBUG::2012-12-09 12:54:28,959::task::588::TaskManager.Task::(_updateState) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::moving from state preparing -> state finished
>Thread-2247::DEBUG::2012-12-09 12:54:28,959::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2247::DEBUG::2012-12-09 12:54:28,959::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2247::DEBUG::2012-12-09 12:54:28,959::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2247::DEBUG::2012-12-09 12:54:28,959::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2247::DEBUG::2012-12-09 12:54:28,960::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2247::DEBUG::2012-12-09 12:54:28,960::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2247::DEBUG::2012-12-09 12:54:28,960::task::978::TaskManager.Task::(_decref) Task=`c44e58c6-3dda-4160-9600-076b1d9beebb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:54:29,535::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:29,576::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016554 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:31,583::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:31,646::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265735 s, 38.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:54:31,698::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:31,708::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000455654 s, 9.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:33,636::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:54:33,637::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:33,637::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:33,655::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:33,703::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166121 s, 61.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:33,737::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:33,738::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:33,739::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:33,739::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:54:35,710::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:35,757::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160495 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:37,764::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:37,812::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177544 s, 57.7 MB/s\n'; <rc> = 0
>Thread-2252::DEBUG::2012-12-09 12:54:38,951::task::588::TaskManager.Task::(_updateState) Task=`9e7d4abc-efa3-4be7-8d84-236dd02382be`::moving from state init -> state preparing
>Thread-2252::INFO::2012-12-09 12:54:38,952::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2252::INFO::2012-12-09 12:54:38,952::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108618736267', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-2252::DEBUG::2012-12-09 12:54:38,952::task::1172::TaskManager.Task::(prepare) Task=`9e7d4abc-efa3-4be7-8d84-236dd02382be`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108618736267', 'lastCheck': '7.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-2252::DEBUG::2012-12-09 12:54:38,953::task::588::TaskManager.Task::(_updateState) Task=`9e7d4abc-efa3-4be7-8d84-236dd02382be`::moving from state preparing -> state finished
>Thread-2252::DEBUG::2012-12-09 12:54:38,953::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2252::DEBUG::2012-12-09 12:54:38,953::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2252::DEBUG::2012-12-09 12:54:38,953::task::978::TaskManager.Task::(_decref) Task=`9e7d4abc-efa3-4be7-8d84-236dd02382be`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:54:39,819::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:39,858::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162925 s, 62.9 MB/s\n'; <rc> = 0
>Thread-2254::DEBUG::2012-12-09 12:54:40,383::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2254::DEBUG::2012-12-09 12:54:40,384::task::588::TaskManager.Task::(_updateState) Task=`1b4c994c-0e15-455d-88ac-c2e9108fabfa`::moving from state init -> state preparing
>Thread-2254::INFO::2012-12-09 12:54:40,384::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2254::INFO::2012-12-09 12:54:40,385::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2254::DEBUG::2012-12-09 12:54:40,385::task::1172::TaskManager.Task::(prepare) Task=`1b4c994c-0e15-455d-88ac-c2e9108fabfa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2254::DEBUG::2012-12-09 12:54:40,385::task::588::TaskManager.Task::(_updateState) Task=`1b4c994c-0e15-455d-88ac-c2e9108fabfa`::moving from state preparing -> state finished
>Thread-2254::DEBUG::2012-12-09 12:54:40,385::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2254::DEBUG::2012-12-09 12:54:40,385::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2254::DEBUG::2012-12-09 12:54:40,386::task::978::TaskManager.Task::(_decref) Task=`1b4c994c-0e15-455d-88ac-c2e9108fabfa`::ref 0 aborting False
>Thread-2255::DEBUG::2012-12-09 12:54:40,398::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2255::DEBUG::2012-12-09 12:54:40,399::task::588::TaskManager.Task::(_updateState) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::moving from state init -> state preparing
>Thread-2255::INFO::2012-12-09 12:54:40,399::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2255::DEBUG::2012-12-09 12:54:40,399::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`48f14447-cd48-404d-ab52-b214a22bcde3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2255::DEBUG::2012-12-09 12:54:40,399::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2255::DEBUG::2012-12-09 12:54:40,400::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2255::DEBUG::2012-12-09 12:54:40,400::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`48f14447-cd48-404d-ab52-b214a22bcde3`::Granted request
>Thread-2255::DEBUG::2012-12-09 12:54:40,400::task::817::TaskManager.Task::(resourceAcquired) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2255::DEBUG::2012-12-09 12:54:40,400::task::978::TaskManager.Task::(_decref) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::ref 1 aborting False
>Thread-2255::DEBUG::2012-12-09 12:54:40,401::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2255::DEBUG::2012-12-09 12:54:40,402::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2255::DEBUG::2012-12-09 12:54:40,498::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2255::WARNING::2012-12-09 12:54:40,499::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2255::DEBUG::2012-12-09 12:54:40,500::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2255::WARNING::2012-12-09 12:54:40,500::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2255::INFO::2012-12-09 12:54:40,501::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2255::DEBUG::2012-12-09 12:54:40,501::task::1172::TaskManager.Task::(prepare) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2255::DEBUG::2012-12-09 12:54:40,501::task::588::TaskManager.Task::(_updateState) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::moving from state preparing -> state finished
>Thread-2255::DEBUG::2012-12-09 12:54:40,501::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2255::DEBUG::2012-12-09 12:54:40,502::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2255::DEBUG::2012-12-09 12:54:40,502::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2255::DEBUG::2012-12-09 12:54:40,502::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2255::DEBUG::2012-12-09 12:54:40,502::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2255::DEBUG::2012-12-09 12:54:40,502::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2255::DEBUG::2012-12-09 12:54:40,503::task::978::TaskManager.Task::(_decref) Task=`a4b95286-c338-46d2-b841-ea2f7935edc7`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:54:41,711::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:41,719::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000378125 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:41,865::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:41,906::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169146 s, 60.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:43,740::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:43,741::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:54:43,839::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:43,840::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:43,840::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:43,841::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:54:43,913::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:43,953::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172426 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:45,960::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:46,000::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165854 s, 61.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:48,007::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:48,048::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017613 s, 58.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:50,055::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:50,102::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237698 s, 43.1 MB/s\n'; <rc> = 0
>Thread-2260::DEBUG::2012-12-09 12:54:50,576::task::588::TaskManager.Task::(_updateState) Task=`4c669ef0-fc93-470d-9648-0f8bd9683741`::moving from state init -> state preparing
>Thread-2260::INFO::2012-12-09 12:54:50,576::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2260::INFO::2012-12-09 12:54:50,576::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00818204879761', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2260::DEBUG::2012-12-09 12:54:50,576::task::1172::TaskManager.Task::(prepare) Task=`4c669ef0-fc93-470d-9648-0f8bd9683741`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00818204879761', 'lastCheck': '8.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2260::DEBUG::2012-12-09 12:54:50,577::task::588::TaskManager.Task::(_updateState) Task=`4c669ef0-fc93-470d-9648-0f8bd9683741`::moving from state preparing -> state finished
>Thread-2260::DEBUG::2012-12-09 12:54:50,577::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2260::DEBUG::2012-12-09 12:54:50,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2260::DEBUG::2012-12-09 12:54:50,577::task::978::TaskManager.Task::(_decref) Task=`4c669ef0-fc93-470d-9648-0f8bd9683741`::ref 0 aborting False
>Thread-2262::DEBUG::2012-12-09 12:54:51,199::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2262::DEBUG::2012-12-09 12:54:51,200::task::588::TaskManager.Task::(_updateState) Task=`3d53a644-0e95-4353-8dd8-a1a002d22c45`::moving from state init -> state preparing
>Thread-2262::INFO::2012-12-09 12:54:51,200::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2262::INFO::2012-12-09 12:54:51,201::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2262::DEBUG::2012-12-09 12:54:51,201::task::1172::TaskManager.Task::(prepare) Task=`3d53a644-0e95-4353-8dd8-a1a002d22c45`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2262::DEBUG::2012-12-09 12:54:51,201::task::588::TaskManager.Task::(_updateState) Task=`3d53a644-0e95-4353-8dd8-a1a002d22c45`::moving from state preparing -> state finished
>Thread-2262::DEBUG::2012-12-09 12:54:51,201::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2262::DEBUG::2012-12-09 12:54:51,201::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2262::DEBUG::2012-12-09 12:54:51,201::task::978::TaskManager.Task::(_decref) Task=`3d53a644-0e95-4353-8dd8-a1a002d22c45`::ref 0 aborting False
>Thread-2263::DEBUG::2012-12-09 12:54:51,222::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2263::DEBUG::2012-12-09 12:54:51,223::task::588::TaskManager.Task::(_updateState) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::moving from state init -> state preparing
>Thread-2263::INFO::2012-12-09 12:54:51,223::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2263::DEBUG::2012-12-09 12:54:51,224::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1d36ba6c-b7be-49cd-9394-24ae1f39fa6d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2263::DEBUG::2012-12-09 12:54:51,224::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2263::DEBUG::2012-12-09 12:54:51,224::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2263::DEBUG::2012-12-09 12:54:51,224::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1d36ba6c-b7be-49cd-9394-24ae1f39fa6d`::Granted request
>Thread-2263::DEBUG::2012-12-09 12:54:51,225::task::817::TaskManager.Task::(resourceAcquired) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2263::DEBUG::2012-12-09 12:54:51,225::task::978::TaskManager.Task::(_decref) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::ref 1 aborting False
>Thread-2263::DEBUG::2012-12-09 12:54:51,225::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2263::DEBUG::2012-12-09 12:54:51,226::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2263::DEBUG::2012-12-09 12:54:51,347::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2263::WARNING::2012-12-09 12:54:51,349::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2263::DEBUG::2012-12-09 12:54:51,349::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2263::WARNING::2012-12-09 12:54:51,349::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2263::INFO::2012-12-09 12:54:51,350::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2263::DEBUG::2012-12-09 12:54:51,350::task::1172::TaskManager.Task::(prepare) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2263::DEBUG::2012-12-09 12:54:51,350::task::588::TaskManager.Task::(_updateState) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::moving from state preparing -> state finished
>Thread-2263::DEBUG::2012-12-09 12:54:51,350::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2263::DEBUG::2012-12-09 12:54:51,351::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2263::DEBUG::2012-12-09 12:54:51,351::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2263::DEBUG::2012-12-09 12:54:51,351::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2263::DEBUG::2012-12-09 12:54:51,351::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2263::DEBUG::2012-12-09 12:54:51,351::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2263::DEBUG::2012-12-09 12:54:51,352::task::978::TaskManager.Task::(_decref) Task=`4178cd7e-3a35-49fc-8724-2ec2583303cd`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:54:51,722::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:54:51,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000656157 s, 6.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:52,109::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:52,150::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155543 s, 65.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:54:53,841::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:54:53,842::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:54:53,940::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:54:53,941::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:54:53,942::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:54:53,942::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:54:54,157::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:54,200::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188185 s, 54.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:56,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:56,245::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161831 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:54:58,252::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:54:58,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173835 s, 58.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:00,300::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:00,347::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244556 s, 41.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:01,738::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:01,751::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000542694 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:02,354::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:02,395::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176556 s, 58.0 MB/s\n'; <rc> = 0
>Thread-2268::DEBUG::2012-12-09 12:55:03,162::task::588::TaskManager.Task::(_updateState) Task=`9086758e-5468-47a5-ab96-d3ff620bc0f3`::moving from state init -> state preparing
>Thread-2268::INFO::2012-12-09 12:55:03,163::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2268::INFO::2012-12-09 12:55:03,163::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137670040131', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-2268::DEBUG::2012-12-09 12:55:03,164::task::1172::TaskManager.Task::(prepare) Task=`9086758e-5468-47a5-ab96-d3ff620bc0f3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0137670040131', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-2268::DEBUG::2012-12-09 12:55:03,164::task::588::TaskManager.Task::(_updateState) Task=`9086758e-5468-47a5-ab96-d3ff620bc0f3`::moving from state preparing -> state finished
>Thread-2268::DEBUG::2012-12-09 12:55:03,164::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2268::DEBUG::2012-12-09 12:55:03,164::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2268::DEBUG::2012-12-09 12:55:03,165::task::978::TaskManager.Task::(_decref) Task=`9086758e-5468-47a5-ab96-d3ff620bc0f3`::ref 0 aborting False
>Thread-2270::DEBUG::2012-12-09 12:55:03,462::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2270::DEBUG::2012-12-09 12:55:03,463::task::588::TaskManager.Task::(_updateState) Task=`b66a1f94-088c-468b-bc00-de0106d00b79`::moving from state init -> state preparing
>Thread-2270::INFO::2012-12-09 12:55:03,463::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2270::INFO::2012-12-09 12:55:03,464::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2270::DEBUG::2012-12-09 12:55:03,464::task::1172::TaskManager.Task::(prepare) Task=`b66a1f94-088c-468b-bc00-de0106d00b79`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2270::DEBUG::2012-12-09 12:55:03,464::task::588::TaskManager.Task::(_updateState) Task=`b66a1f94-088c-468b-bc00-de0106d00b79`::moving from state preparing -> state finished
>Thread-2270::DEBUG::2012-12-09 12:55:03,464::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2270::DEBUG::2012-12-09 12:55:03,464::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2270::DEBUG::2012-12-09 12:55:03,464::task::978::TaskManager.Task::(_decref) Task=`b66a1f94-088c-468b-bc00-de0106d00b79`::ref 0 aborting False
>Thread-2271::DEBUG::2012-12-09 12:55:03,479::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2271::DEBUG::2012-12-09 12:55:03,479::task::588::TaskManager.Task::(_updateState) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::moving from state init -> state preparing
>Thread-2271::INFO::2012-12-09 12:55:03,480::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2271::DEBUG::2012-12-09 12:55:03,480::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1d92539a-7a14-4382-883a-5c7f16f9d8c5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2271::DEBUG::2012-12-09 12:55:03,480::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2271::DEBUG::2012-12-09 12:55:03,480::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2271::DEBUG::2012-12-09 12:55:03,481::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1d92539a-7a14-4382-883a-5c7f16f9d8c5`::Granted request
>Thread-2271::DEBUG::2012-12-09 12:55:03,481::task::817::TaskManager.Task::(resourceAcquired) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2271::DEBUG::2012-12-09 12:55:03,481::task::978::TaskManager.Task::(_decref) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::ref 1 aborting False
>Thread-2271::DEBUG::2012-12-09 12:55:03,482::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2271::DEBUG::2012-12-09 12:55:03,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2271::DEBUG::2012-12-09 12:55:03,580::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2271::WARNING::2012-12-09 12:55:03,582::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2271::DEBUG::2012-12-09 12:55:03,582::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2271::WARNING::2012-12-09 12:55:03,582::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2271::INFO::2012-12-09 12:55:03,583::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2271::DEBUG::2012-12-09 12:55:03,583::task::1172::TaskManager.Task::(prepare) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2271::DEBUG::2012-12-09 12:55:03,583::task::588::TaskManager.Task::(_updateState) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::moving from state preparing -> state finished
>Thread-2271::DEBUG::2012-12-09 12:55:03,583::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2271::DEBUG::2012-12-09 12:55:03,584::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2271::DEBUG::2012-12-09 12:55:03,584::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2271::DEBUG::2012-12-09 12:55:03,584::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2271::DEBUG::2012-12-09 12:55:03,584::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2271::DEBUG::2012-12-09 12:55:03,584::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2271::DEBUG::2012-12-09 12:55:03,585::task::978::TaskManager.Task::(_decref) Task=`0bf9d9a6-388b-456c-ad95-3c0148dd40f2`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:55:03,943::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:03,944::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:55:04,044::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:04,045::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:04,046::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:04,046::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:55:04,402::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:04,442::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167104 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:06,449::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:06,489::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170659 s, 60.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:08,496::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:08,537::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190718 s, 53.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:10,544::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:10,584::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164582 s, 62.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:11,754::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:11,762::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00046874 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:12,591::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:12,634::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170148 s, 60.2 MB/s\n'; <rc> = 0
>Thread-2276::DEBUG::2012-12-09 12:55:13,926::task::588::TaskManager.Task::(_updateState) Task=`843c7e86-fa34-43d6-8c0d-ac71e601dfea`::moving from state init -> state preparing
>Thread-2276::INFO::2012-12-09 12:55:13,926::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2276::INFO::2012-12-09 12:55:13,927::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839591026306', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2276::DEBUG::2012-12-09 12:55:13,927::task::1172::TaskManager.Task::(prepare) Task=`843c7e86-fa34-43d6-8c0d-ac71e601dfea`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839591026306', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2276::DEBUG::2012-12-09 12:55:13,927::task::588::TaskManager.Task::(_updateState) Task=`843c7e86-fa34-43d6-8c0d-ac71e601dfea`::moving from state preparing -> state finished
>Thread-2276::DEBUG::2012-12-09 12:55:13,927::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2276::DEBUG::2012-12-09 12:55:13,928::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2276::DEBUG::2012-12-09 12:55:13,928::task::978::TaskManager.Task::(_decref) Task=`843c7e86-fa34-43d6-8c0d-ac71e601dfea`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:55:14,047::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:14,048::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2278::DEBUG::2012-12-09 12:55:14,071::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2278::DEBUG::2012-12-09 12:55:14,071::task::588::TaskManager.Task::(_updateState) Task=`96215c9e-b8d5-4baf-bbb4-90d3b7009a6a`::moving from state init -> state preparing
>Thread-2278::INFO::2012-12-09 12:55:14,072::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2278::INFO::2012-12-09 12:55:14,072::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2278::DEBUG::2012-12-09 12:55:14,072::task::1172::TaskManager.Task::(prepare) Task=`96215c9e-b8d5-4baf-bbb4-90d3b7009a6a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2278::DEBUG::2012-12-09 12:55:14,072::task::588::TaskManager.Task::(_updateState) Task=`96215c9e-b8d5-4baf-bbb4-90d3b7009a6a`::moving from state preparing -> state finished
>Thread-2278::DEBUG::2012-12-09 12:55:14,073::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2278::DEBUG::2012-12-09 12:55:14,073::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2278::DEBUG::2012-12-09 12:55:14,073::task::978::TaskManager.Task::(_decref) Task=`96215c9e-b8d5-4baf-bbb4-90d3b7009a6a`::ref 0 aborting False
>Thread-2279::DEBUG::2012-12-09 12:55:14,087::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2279::DEBUG::2012-12-09 12:55:14,087::task::588::TaskManager.Task::(_updateState) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::moving from state init -> state preparing
>Thread-2279::INFO::2012-12-09 12:55:14,087::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2279::DEBUG::2012-12-09 12:55:14,088::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`95d786d6-4e17-4d11-9232-20714d25747b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2279::DEBUG::2012-12-09 12:55:14,088::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2279::DEBUG::2012-12-09 12:55:14,088::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2279::DEBUG::2012-12-09 12:55:14,089::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`95d786d6-4e17-4d11-9232-20714d25747b`::Granted request
>Thread-2279::DEBUG::2012-12-09 12:55:14,089::task::817::TaskManager.Task::(resourceAcquired) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2279::DEBUG::2012-12-09 12:55:14,089::task::978::TaskManager.Task::(_decref) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 12:55:14,164::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:14,166::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:14,167::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:14,168::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2279::DEBUG::2012-12-09 12:55:14,169::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2279::DEBUG::2012-12-09 12:55:14,170::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2279::DEBUG::2012-12-09 12:55:14,276::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2279::WARNING::2012-12-09 12:55:14,277::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2279::DEBUG::2012-12-09 12:55:14,277::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2279::WARNING::2012-12-09 12:55:14,278::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2279::INFO::2012-12-09 12:55:14,278::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2279::DEBUG::2012-12-09 12:55:14,278::task::1172::TaskManager.Task::(prepare) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2279::DEBUG::2012-12-09 12:55:14,279::task::588::TaskManager.Task::(_updateState) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::moving from state preparing -> state finished
>Thread-2279::DEBUG::2012-12-09 12:55:14,279::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2279::DEBUG::2012-12-09 12:55:14,279::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2279::DEBUG::2012-12-09 12:55:14,279::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2279::DEBUG::2012-12-09 12:55:14,279::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2279::DEBUG::2012-12-09 12:55:14,280::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2279::DEBUG::2012-12-09 12:55:14,280::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2279::DEBUG::2012-12-09 12:55:14,280::task::978::TaskManager.Task::(_decref) Task=`d308d46f-25f3-44fe-9202-4db5a6df5a60`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:14,641::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:14,688::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019007 s, 53.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:16,704::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:16,746::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168174 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:18,753::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:18,793::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172296 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:20,800::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:20,840::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171185 s, 59.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:21,765::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:21,773::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000544759 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:22,847::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:22,887::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166103 s, 61.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:55:24,169::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:24,170::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:55:24,278::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:24,279::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:24,279::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:24,280::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2284::DEBUG::2012-12-09 12:55:24,606::task::588::TaskManager.Task::(_updateState) Task=`4c77d890-51c8-485e-b643-2e69bdf900c8`::moving from state init -> state preparing
>Thread-2284::INFO::2012-12-09 12:55:24,606::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2284::INFO::2012-12-09 12:55:24,607::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839400291443', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-2284::DEBUG::2012-12-09 12:55:24,607::task::1172::TaskManager.Task::(prepare) Task=`4c77d890-51c8-485e-b643-2e69bdf900c8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839400291443', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.3', 'code': 358, 'valid': False}}
>Thread-2284::DEBUG::2012-12-09 12:55:24,607::task::588::TaskManager.Task::(_updateState) Task=`4c77d890-51c8-485e-b643-2e69bdf900c8`::moving from state preparing -> state finished
>Thread-2284::DEBUG::2012-12-09 12:55:24,607::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2284::DEBUG::2012-12-09 12:55:24,607::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2284::DEBUG::2012-12-09 12:55:24,608::task::978::TaskManager.Task::(_decref) Task=`4c77d890-51c8-485e-b643-2e69bdf900c8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:24,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:24,933::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163236 s, 62.7 MB/s\n'; <rc> = 0
>Thread-2286::DEBUG::2012-12-09 12:55:25,536::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2286::DEBUG::2012-12-09 12:55:25,537::task::588::TaskManager.Task::(_updateState) Task=`ea0ad3af-0e52-472b-81ba-540a940d1887`::moving from state init -> state preparing
>Thread-2286::INFO::2012-12-09 12:55:25,537::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2286::INFO::2012-12-09 12:55:25,537::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2286::DEBUG::2012-12-09 12:55:25,538::task::1172::TaskManager.Task::(prepare) Task=`ea0ad3af-0e52-472b-81ba-540a940d1887`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2286::DEBUG::2012-12-09 12:55:25,538::task::588::TaskManager.Task::(_updateState) Task=`ea0ad3af-0e52-472b-81ba-540a940d1887`::moving from state preparing -> state finished
>Thread-2286::DEBUG::2012-12-09 12:55:25,538::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2286::DEBUG::2012-12-09 12:55:25,538::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2286::DEBUG::2012-12-09 12:55:25,538::task::978::TaskManager.Task::(_decref) Task=`ea0ad3af-0e52-472b-81ba-540a940d1887`::ref 0 aborting False
>Thread-2287::DEBUG::2012-12-09 12:55:25,551::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2287::DEBUG::2012-12-09 12:55:25,551::task::588::TaskManager.Task::(_updateState) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::moving from state init -> state preparing
>Thread-2287::INFO::2012-12-09 12:55:25,551::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2287::DEBUG::2012-12-09 12:55:25,552::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fe0e1ab2-808b-4bc2-9bbe-db1085ec8cc1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2287::DEBUG::2012-12-09 12:55:25,552::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2287::DEBUG::2012-12-09 12:55:25,552::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2287::DEBUG::2012-12-09 12:55:25,552::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fe0e1ab2-808b-4bc2-9bbe-db1085ec8cc1`::Granted request
>Thread-2287::DEBUG::2012-12-09 12:55:25,553::task::817::TaskManager.Task::(resourceAcquired) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2287::DEBUG::2012-12-09 12:55:25,553::task::978::TaskManager.Task::(_decref) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::ref 1 aborting False
>Thread-2287::DEBUG::2012-12-09 12:55:25,554::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2287::DEBUG::2012-12-09 12:55:25,554::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2287::DEBUG::2012-12-09 12:55:25,652::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2287::WARNING::2012-12-09 12:55:25,653::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2287::DEBUG::2012-12-09 12:55:25,654::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2287::WARNING::2012-12-09 12:55:25,654::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2287::INFO::2012-12-09 12:55:25,654::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2287::DEBUG::2012-12-09 12:55:25,655::task::1172::TaskManager.Task::(prepare) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2287::DEBUG::2012-12-09 12:55:25,655::task::588::TaskManager.Task::(_updateState) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::moving from state preparing -> state finished
>Thread-2287::DEBUG::2012-12-09 12:55:25,655::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2287::DEBUG::2012-12-09 12:55:25,655::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2287::DEBUG::2012-12-09 12:55:25,655::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2287::DEBUG::2012-12-09 12:55:25,656::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2287::DEBUG::2012-12-09 12:55:25,656::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2287::DEBUG::2012-12-09 12:55:25,656::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2287::DEBUG::2012-12-09 12:55:25,656::task::978::TaskManager.Task::(_decref) Task=`4d061a3e-a3ad-4b89-996e-00646f75e314`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:26,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:26,979::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161731 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:28,986::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:29,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171209 s, 59.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:31,032::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:31,073::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165978 s, 61.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:31,776::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:31,787::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000561831 s, 7.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:33,079::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:33,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163293 s, 62.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:55:34,280::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:34,281::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:55:34,378::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:34,379::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:34,380::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:34,380::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:55:35,126::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:35,166::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170529 s, 60.0 MB/s\n'; <rc> = 0
>Thread-2292::DEBUG::2012-12-09 12:55:35,565::task::588::TaskManager.Task::(_updateState) Task=`6436d059-5eda-4b9c-9367-edf678dff4d3`::moving from state init -> state preparing
>Thread-2292::INFO::2012-12-09 12:55:35,566::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2292::INFO::2012-12-09 12:55:35,566::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118370056152', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-2292::DEBUG::2012-12-09 12:55:35,566::task::1172::TaskManager.Task::(prepare) Task=`6436d059-5eda-4b9c-9367-edf678dff4d3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118370056152', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.2', 'code': 358, 'valid': False}}
>Thread-2292::DEBUG::2012-12-09 12:55:35,566::task::588::TaskManager.Task::(_updateState) Task=`6436d059-5eda-4b9c-9367-edf678dff4d3`::moving from state preparing -> state finished
>Thread-2292::DEBUG::2012-12-09 12:55:35,567::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2292::DEBUG::2012-12-09 12:55:35,567::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2292::DEBUG::2012-12-09 12:55:35,567::task::978::TaskManager.Task::(_decref) Task=`6436d059-5eda-4b9c-9367-edf678dff4d3`::ref 0 aborting False
>Thread-2294::DEBUG::2012-12-09 12:55:36,331::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2294::DEBUG::2012-12-09 12:55:36,331::task::588::TaskManager.Task::(_updateState) Task=`9a3c0931-f9b6-4132-9647-eb09482ae8b9`::moving from state init -> state preparing
>Thread-2294::INFO::2012-12-09 12:55:36,331::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2294::INFO::2012-12-09 12:55:36,332::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2294::DEBUG::2012-12-09 12:55:36,332::task::1172::TaskManager.Task::(prepare) Task=`9a3c0931-f9b6-4132-9647-eb09482ae8b9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2294::DEBUG::2012-12-09 12:55:36,332::task::588::TaskManager.Task::(_updateState) Task=`9a3c0931-f9b6-4132-9647-eb09482ae8b9`::moving from state preparing -> state finished
>Thread-2294::DEBUG::2012-12-09 12:55:36,332::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2294::DEBUG::2012-12-09 12:55:36,332::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2294::DEBUG::2012-12-09 12:55:36,333::task::978::TaskManager.Task::(_decref) Task=`9a3c0931-f9b6-4132-9647-eb09482ae8b9`::ref 0 aborting False
>Thread-2295::DEBUG::2012-12-09 12:55:36,347::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2295::DEBUG::2012-12-09 12:55:36,347::task::588::TaskManager.Task::(_updateState) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::moving from state init -> state preparing
>Thread-2295::INFO::2012-12-09 12:55:36,348::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2295::DEBUG::2012-12-09 12:55:36,348::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3d2af6ce-0b7d-40a7-a829-42b977a2d64f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2295::DEBUG::2012-12-09 12:55:36,348::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2295::DEBUG::2012-12-09 12:55:36,348::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2295::DEBUG::2012-12-09 12:55:36,349::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3d2af6ce-0b7d-40a7-a829-42b977a2d64f`::Granted request
>Thread-2295::DEBUG::2012-12-09 12:55:36,349::task::817::TaskManager.Task::(resourceAcquired) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2295::DEBUG::2012-12-09 12:55:36,349::task::978::TaskManager.Task::(_decref) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::ref 1 aborting False
>Thread-2295::DEBUG::2012-12-09 12:55:36,350::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2295::DEBUG::2012-12-09 12:55:36,350::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2295::DEBUG::2012-12-09 12:55:36,450::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2295::WARNING::2012-12-09 12:55:36,451::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2295::DEBUG::2012-12-09 12:55:36,452::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2295::WARNING::2012-12-09 12:55:36,452::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2295::INFO::2012-12-09 12:55:36,452::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2295::DEBUG::2012-12-09 12:55:36,453::task::1172::TaskManager.Task::(prepare) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2295::DEBUG::2012-12-09 12:55:36,453::task::588::TaskManager.Task::(_updateState) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::moving from state preparing -> state finished
>Thread-2295::DEBUG::2012-12-09 12:55:36,453::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2295::DEBUG::2012-12-09 12:55:36,453::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2295::DEBUG::2012-12-09 12:55:36,454::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2295::DEBUG::2012-12-09 12:55:36,454::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2295::DEBUG::2012-12-09 12:55:36,454::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2295::DEBUG::2012-12-09 12:55:36,454::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2295::DEBUG::2012-12-09 12:55:36,454::task::978::TaskManager.Task::(_decref) Task=`bbcabbdc-7c50-4fd7-935c-061c0a74e66d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:37,173::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:37,212::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156639 s, 65.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:39,219::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:39,260::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174996 s, 58.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:41,267::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:41,307::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181758 s, 56.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:41,791::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:41,802::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00045345 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:43,314::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:43,357::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169669 s, 60.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:55:44,381::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:44,382::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:55:44,491::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:44,492::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:44,493::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:44,493::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:55:45,364::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:45,404::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169586 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:47,411::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:47,451::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173365 s, 59.1 MB/s\n'; <rc> = 0
>Thread-2300::DEBUG::2012-12-09 12:55:47,984::task::588::TaskManager.Task::(_updateState) Task=`e344eca5-353d-4186-a083-ed9584c96bfd`::moving from state init -> state preparing
>Thread-2300::INFO::2012-12-09 12:55:47,984::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2300::INFO::2012-12-09 12:55:47,985::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116629600525', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-2300::DEBUG::2012-12-09 12:55:47,985::task::1172::TaskManager.Task::(prepare) Task=`e344eca5-353d-4186-a083-ed9584c96bfd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116629600525', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-2300::DEBUG::2012-12-09 12:55:47,985::task::588::TaskManager.Task::(_updateState) Task=`e344eca5-353d-4186-a083-ed9584c96bfd`::moving from state preparing -> state finished
>Thread-2300::DEBUG::2012-12-09 12:55:47,985::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2300::DEBUG::2012-12-09 12:55:47,986::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2300::DEBUG::2012-12-09 12:55:47,986::task::978::TaskManager.Task::(_decref) Task=`e344eca5-353d-4186-a083-ed9584c96bfd`::ref 0 aborting False
>Thread-2302::DEBUG::2012-12-09 12:55:48,255::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2302::DEBUG::2012-12-09 12:55:48,255::task::588::TaskManager.Task::(_updateState) Task=`b5e45683-f8f2-43bb-94bf-0d73135860b2`::moving from state init -> state preparing
>Thread-2302::INFO::2012-12-09 12:55:48,255::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2302::INFO::2012-12-09 12:55:48,256::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2302::DEBUG::2012-12-09 12:55:48,256::task::1172::TaskManager.Task::(prepare) Task=`b5e45683-f8f2-43bb-94bf-0d73135860b2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2302::DEBUG::2012-12-09 12:55:48,256::task::588::TaskManager.Task::(_updateState) Task=`b5e45683-f8f2-43bb-94bf-0d73135860b2`::moving from state preparing -> state finished
>Thread-2302::DEBUG::2012-12-09 12:55:48,256::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2302::DEBUG::2012-12-09 12:55:48,256::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2302::DEBUG::2012-12-09 12:55:48,257::task::978::TaskManager.Task::(_decref) Task=`b5e45683-f8f2-43bb-94bf-0d73135860b2`::ref 0 aborting False
>Thread-2303::DEBUG::2012-12-09 12:55:48,272::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2303::DEBUG::2012-12-09 12:55:48,273::task::588::TaskManager.Task::(_updateState) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::moving from state init -> state preparing
>Thread-2303::INFO::2012-12-09 12:55:48,273::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2303::DEBUG::2012-12-09 12:55:48,273::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`516ed979-7539-4e66-962e-113790178150`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2303::DEBUG::2012-12-09 12:55:48,273::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2303::DEBUG::2012-12-09 12:55:48,274::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2303::DEBUG::2012-12-09 12:55:48,274::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`516ed979-7539-4e66-962e-113790178150`::Granted request
>Thread-2303::DEBUG::2012-12-09 12:55:48,274::task::817::TaskManager.Task::(resourceAcquired) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2303::DEBUG::2012-12-09 12:55:48,274::task::978::TaskManager.Task::(_decref) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::ref 1 aborting False
>Thread-2303::DEBUG::2012-12-09 12:55:48,275::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2303::DEBUG::2012-12-09 12:55:48,276::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2303::DEBUG::2012-12-09 12:55:48,374::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2303::WARNING::2012-12-09 12:55:48,375::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2303::DEBUG::2012-12-09 12:55:48,375::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2303::WARNING::2012-12-09 12:55:48,376::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2303::INFO::2012-12-09 12:55:48,376::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2303::DEBUG::2012-12-09 12:55:48,376::task::1172::TaskManager.Task::(prepare) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2303::DEBUG::2012-12-09 12:55:48,377::task::588::TaskManager.Task::(_updateState) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::moving from state preparing -> state finished
>Thread-2303::DEBUG::2012-12-09 12:55:48,377::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2303::DEBUG::2012-12-09 12:55:48,377::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2303::DEBUG::2012-12-09 12:55:48,377::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2303::DEBUG::2012-12-09 12:55:48,377::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2303::DEBUG::2012-12-09 12:55:48,378::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2303::DEBUG::2012-12-09 12:55:48,378::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2303::DEBUG::2012-12-09 12:55:48,378::task::978::TaskManager.Task::(_decref) Task=`ec551bd1-3c5c-4dfb-ac2d-005b1df9605e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:49,458::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:49,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163986 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:51,505::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:51,547::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187392 s, 54.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:55:51,805::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:55:51,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000387804 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:53,552::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:53,595::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179439 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:55:54,494::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:55:54,495::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:55:54,597::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:55:54,598::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:55:54,598::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:55:54,599::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:55:55,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:55,643::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161883 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:55:57,650::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:57,688::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160306 s, 63.9 MB/s\n'; <rc> = 0
>Thread-2308::DEBUG::2012-12-09 12:55:59,476::task::588::TaskManager.Task::(_updateState) Task=`8cde3f79-dfa2-4483-9443-4c979334774c`::moving from state init -> state preparing
>Thread-2308::INFO::2012-12-09 12:55:59,476::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2308::INFO::2012-12-09 12:55:59,476::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129599571228', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2308::DEBUG::2012-12-09 12:55:59,477::task::1172::TaskManager.Task::(prepare) Task=`8cde3f79-dfa2-4483-9443-4c979334774c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129599571228', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2308::DEBUG::2012-12-09 12:55:59,477::task::588::TaskManager.Task::(_updateState) Task=`8cde3f79-dfa2-4483-9443-4c979334774c`::moving from state preparing -> state finished
>Thread-2308::DEBUG::2012-12-09 12:55:59,477::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2308::DEBUG::2012-12-09 12:55:59,477::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2308::DEBUG::2012-12-09 12:55:59,477::task::978::TaskManager.Task::(_decref) Task=`8cde3f79-dfa2-4483-9443-4c979334774c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:55:59,694::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:55:59,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161317 s, 63.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:01,741::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:01,782::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173826 s, 58.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:56:01,820::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:01,828::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000457199 s, 9.0 MB/s\n'; <rc> = 0
>Thread-2311::DEBUG::2012-12-09 12:56:02,702::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2311::DEBUG::2012-12-09 12:56:02,703::task::588::TaskManager.Task::(_updateState) Task=`a7032009-28e7-4629-937a-cd347575bcf8`::moving from state init -> state preparing
>Thread-2311::INFO::2012-12-09 12:56:02,703::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2311::INFO::2012-12-09 12:56:02,703::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2311::DEBUG::2012-12-09 12:56:02,703::task::1172::TaskManager.Task::(prepare) Task=`a7032009-28e7-4629-937a-cd347575bcf8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2311::DEBUG::2012-12-09 12:56:02,704::task::588::TaskManager.Task::(_updateState) Task=`a7032009-28e7-4629-937a-cd347575bcf8`::moving from state preparing -> state finished
>Thread-2311::DEBUG::2012-12-09 12:56:02,704::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2311::DEBUG::2012-12-09 12:56:02,704::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2311::DEBUG::2012-12-09 12:56:02,704::task::978::TaskManager.Task::(_decref) Task=`a7032009-28e7-4629-937a-cd347575bcf8`::ref 0 aborting False
>Thread-2312::DEBUG::2012-12-09 12:56:02,717::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2312::DEBUG::2012-12-09 12:56:02,717::task::588::TaskManager.Task::(_updateState) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::moving from state init -> state preparing
>Thread-2312::INFO::2012-12-09 12:56:02,717::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2312::DEBUG::2012-12-09 12:56:02,718::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6c07f6dd-b861-476e-9dfb-dab5d4601651`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2312::DEBUG::2012-12-09 12:56:02,718::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2312::DEBUG::2012-12-09 12:56:02,718::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2312::DEBUG::2012-12-09 12:56:02,718::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6c07f6dd-b861-476e-9dfb-dab5d4601651`::Granted request
>Thread-2312::DEBUG::2012-12-09 12:56:02,719::task::817::TaskManager.Task::(resourceAcquired) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2312::DEBUG::2012-12-09 12:56:02,719::task::978::TaskManager.Task::(_decref) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::ref 1 aborting False
>Thread-2312::DEBUG::2012-12-09 12:56:02,720::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2312::DEBUG::2012-12-09 12:56:02,721::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2312::DEBUG::2012-12-09 12:56:02,820::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2312::WARNING::2012-12-09 12:56:02,822::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2312::DEBUG::2012-12-09 12:56:02,822::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2312::WARNING::2012-12-09 12:56:02,822::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2312::INFO::2012-12-09 12:56:02,823::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2312::DEBUG::2012-12-09 12:56:02,823::task::1172::TaskManager.Task::(prepare) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2312::DEBUG::2012-12-09 12:56:02,823::task::588::TaskManager.Task::(_updateState) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::moving from state preparing -> state finished
>Thread-2312::DEBUG::2012-12-09 12:56:02,823::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2312::DEBUG::2012-12-09 12:56:02,824::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2312::DEBUG::2012-12-09 12:56:02,824::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2312::DEBUG::2012-12-09 12:56:02,824::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2312::DEBUG::2012-12-09 12:56:02,824::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2312::DEBUG::2012-12-09 12:56:02,824::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2312::DEBUG::2012-12-09 12:56:02,825::task::978::TaskManager.Task::(_decref) Task=`682c8a1e-f83d-4457-99d8-eca43d6664cd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:03,789::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:03,839::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255396 s, 40.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:56:04,600::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:04,600::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:04,709::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:04,710::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:04,710::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:04,711::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:56:05,846::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:05,886::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165591 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:07,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:07,933::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168223 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:09,940::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:09,981::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169231 s, 60.5 MB/s\n'; <rc> = 0
>Thread-2316::DEBUG::2012-12-09 12:56:10,234::task::588::TaskManager.Task::(_updateState) Task=`4fb93322-518b-4d85-8b51-cfbd12a6b630`::moving from state init -> state preparing
>Thread-2316::INFO::2012-12-09 12:56:10,235::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2316::INFO::2012-12-09 12:56:10,235::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00828003883362', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-2316::DEBUG::2012-12-09 12:56:10,235::task::1172::TaskManager.Task::(prepare) Task=`4fb93322-518b-4d85-8b51-cfbd12a6b630`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00828003883362', 'lastCheck': '8.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-2316::DEBUG::2012-12-09 12:56:10,236::task::588::TaskManager.Task::(_updateState) Task=`4fb93322-518b-4d85-8b51-cfbd12a6b630`::moving from state preparing -> state finished
>Thread-2316::DEBUG::2012-12-09 12:56:10,236::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2316::DEBUG::2012-12-09 12:56:10,236::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2316::DEBUG::2012-12-09 12:56:10,236::task::978::TaskManager.Task::(_decref) Task=`4fb93322-518b-4d85-8b51-cfbd12a6b630`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:56:11,831::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:11,842::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000346221 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:11,988::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:12,027::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159169 s, 64.3 MB/s\n'; <rc> = 0
>Thread-2319::DEBUG::2012-12-09 12:56:13,629::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2319::DEBUG::2012-12-09 12:56:13,629::task::588::TaskManager.Task::(_updateState) Task=`5077b7d9-cb73-44b8-b068-2f7f9a2d01ac`::moving from state init -> state preparing
>Thread-2319::INFO::2012-12-09 12:56:13,630::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2319::INFO::2012-12-09 12:56:13,630::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2319::DEBUG::2012-12-09 12:56:13,630::task::1172::TaskManager.Task::(prepare) Task=`5077b7d9-cb73-44b8-b068-2f7f9a2d01ac`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2319::DEBUG::2012-12-09 12:56:13,631::task::588::TaskManager.Task::(_updateState) Task=`5077b7d9-cb73-44b8-b068-2f7f9a2d01ac`::moving from state preparing -> state finished
>Thread-2319::DEBUG::2012-12-09 12:56:13,631::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2319::DEBUG::2012-12-09 12:56:13,631::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2319::DEBUG::2012-12-09 12:56:13,631::task::978::TaskManager.Task::(_decref) Task=`5077b7d9-cb73-44b8-b068-2f7f9a2d01ac`::ref 0 aborting False
>Thread-2320::DEBUG::2012-12-09 12:56:13,644::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2320::DEBUG::2012-12-09 12:56:13,644::task::588::TaskManager.Task::(_updateState) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::moving from state init -> state preparing
>Thread-2320::INFO::2012-12-09 12:56:13,645::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2320::DEBUG::2012-12-09 12:56:13,645::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4635e342-a988-4068-990d-d279111cc267`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2320::DEBUG::2012-12-09 12:56:13,646::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2320::DEBUG::2012-12-09 12:56:13,646::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2320::DEBUG::2012-12-09 12:56:13,646::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4635e342-a988-4068-990d-d279111cc267`::Granted request
>Thread-2320::DEBUG::2012-12-09 12:56:13,647::task::817::TaskManager.Task::(resourceAcquired) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2320::DEBUG::2012-12-09 12:56:13,647::task::978::TaskManager.Task::(_decref) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::ref 1 aborting False
>Thread-2320::DEBUG::2012-12-09 12:56:13,648::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2320::DEBUG::2012-12-09 12:56:13,649::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2320::DEBUG::2012-12-09 12:56:13,756::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2320::WARNING::2012-12-09 12:56:13,758::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2320::DEBUG::2012-12-09 12:56:13,758::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2320::WARNING::2012-12-09 12:56:13,759::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2320::INFO::2012-12-09 12:56:13,759::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2320::DEBUG::2012-12-09 12:56:13,760::task::1172::TaskManager.Task::(prepare) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2320::DEBUG::2012-12-09 12:56:13,760::task::588::TaskManager.Task::(_updateState) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::moving from state preparing -> state finished
>Thread-2320::DEBUG::2012-12-09 12:56:13,760::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2320::DEBUG::2012-12-09 12:56:13,761::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2320::DEBUG::2012-12-09 12:56:13,761::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2320::DEBUG::2012-12-09 12:56:13,761::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2320::DEBUG::2012-12-09 12:56:13,762::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2320::DEBUG::2012-12-09 12:56:13,762::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2320::DEBUG::2012-12-09 12:56:13,762::task::978::TaskManager.Task::(_decref) Task=`41cf33bb-a133-495f-b294-8846b81c8de4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:14,034::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:14,074::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162849 s, 62.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:56:14,711::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:14,712::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:14,810::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:14,811::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:14,812::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:14,812::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:56:16,082::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:16,121::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165813 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:18,128::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:18,172::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195554 s, 52.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:20,179::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:20,221::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180109 s, 56.9 MB/s\n'; <rc> = 0
>Thread-2324::DEBUG::2012-12-09 12:56:21,554::task::588::TaskManager.Task::(_updateState) Task=`e4ccdeef-bc0c-4707-ac7c-5b28d6cce739`::moving from state init -> state preparing
>Thread-2324::INFO::2012-12-09 12:56:21,554::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2324::INFO::2012-12-09 12:56:21,555::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116360187531', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2324::DEBUG::2012-12-09 12:56:21,555::task::1172::TaskManager.Task::(prepare) Task=`e4ccdeef-bc0c-4707-ac7c-5b28d6cce739`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116360187531', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2324::DEBUG::2012-12-09 12:56:21,556::task::588::TaskManager.Task::(_updateState) Task=`e4ccdeef-bc0c-4707-ac7c-5b28d6cce739`::moving from state preparing -> state finished
>Thread-2324::DEBUG::2012-12-09 12:56:21,556::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2324::DEBUG::2012-12-09 12:56:21,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2324::DEBUG::2012-12-09 12:56:21,557::task::978::TaskManager.Task::(_decref) Task=`e4ccdeef-bc0c-4707-ac7c-5b28d6cce739`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:56:21,845::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:21,853::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000447872 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:22,228::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:22,268::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159414 s, 64.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:24,275::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:24,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172984 s, 59.2 MB/s\n'; <rc> = 0
>Thread-2326::DEBUG::2012-12-09 12:56:24,361::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2326::DEBUG::2012-12-09 12:56:24,362::task::588::TaskManager.Task::(_updateState) Task=`b9111259-22c1-4cb4-9680-54e4adbea39b`::moving from state init -> state preparing
>Thread-2326::INFO::2012-12-09 12:56:24,362::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2326::INFO::2012-12-09 12:56:24,362::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2326::DEBUG::2012-12-09 12:56:24,362::task::1172::TaskManager.Task::(prepare) Task=`b9111259-22c1-4cb4-9680-54e4adbea39b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2326::DEBUG::2012-12-09 12:56:24,362::task::588::TaskManager.Task::(_updateState) Task=`b9111259-22c1-4cb4-9680-54e4adbea39b`::moving from state preparing -> state finished
>Thread-2326::DEBUG::2012-12-09 12:56:24,363::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2326::DEBUG::2012-12-09 12:56:24,363::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2326::DEBUG::2012-12-09 12:56:24,363::task::978::TaskManager.Task::(_decref) Task=`b9111259-22c1-4cb4-9680-54e4adbea39b`::ref 0 aborting False
>Thread-2327::DEBUG::2012-12-09 12:56:24,377::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2327::DEBUG::2012-12-09 12:56:24,377::task::588::TaskManager.Task::(_updateState) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::moving from state init -> state preparing
>Thread-2327::INFO::2012-12-09 12:56:24,378::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2327::DEBUG::2012-12-09 12:56:24,378::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a6936c9-1d9a-494b-87d3-73fe133ab683`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2327::DEBUG::2012-12-09 12:56:24,378::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2327::DEBUG::2012-12-09 12:56:24,378::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2327::DEBUG::2012-12-09 12:56:24,379::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a6936c9-1d9a-494b-87d3-73fe133ab683`::Granted request
>Thread-2327::DEBUG::2012-12-09 12:56:24,379::task::817::TaskManager.Task::(resourceAcquired) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2327::DEBUG::2012-12-09 12:56:24,379::task::978::TaskManager.Task::(_decref) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::ref 1 aborting False
>Thread-2327::DEBUG::2012-12-09 12:56:24,380::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2327::DEBUG::2012-12-09 12:56:24,380::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2327::DEBUG::2012-12-09 12:56:24,512::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2327::WARNING::2012-12-09 12:56:24,513::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2327::DEBUG::2012-12-09 12:56:24,513::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2327::WARNING::2012-12-09 12:56:24,514::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2327::INFO::2012-12-09 12:56:24,514::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2327::DEBUG::2012-12-09 12:56:24,514::task::1172::TaskManager.Task::(prepare) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2327::DEBUG::2012-12-09 12:56:24,515::task::588::TaskManager.Task::(_updateState) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::moving from state preparing -> state finished
>Thread-2327::DEBUG::2012-12-09 12:56:24,515::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2327::DEBUG::2012-12-09 12:56:24,515::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2327::DEBUG::2012-12-09 12:56:24,515::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2327::DEBUG::2012-12-09 12:56:24,515::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2327::DEBUG::2012-12-09 12:56:24,516::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2327::DEBUG::2012-12-09 12:56:24,516::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2327::DEBUG::2012-12-09 12:56:24,516::task::978::TaskManager.Task::(_decref) Task=`f53a3ed3-e060-4d5c-a059-48d0b508a9af`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:56:24,813::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:24,814::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:24,922::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:24,923::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:24,923::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:24,924::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:56:26,322::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:26,361::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165088 s, 62.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:28,368::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:28,407::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167916 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:30,414::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:30,457::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018142 s, 56.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:56:31,856::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:31,867::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000357328 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:32,464::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:32,505::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174557 s, 58.7 MB/s\n'; <rc> = 0
>Thread-2332::DEBUG::2012-12-09 12:56:32,568::task::588::TaskManager.Task::(_updateState) Task=`aa006a8e-65f1-4d94-9269-831d134501c1`::moving from state init -> state preparing
>Thread-2332::INFO::2012-12-09 12:56:32,569::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2332::INFO::2012-12-09 12:56:32,569::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117361545563', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2332::DEBUG::2012-12-09 12:56:32,569::task::1172::TaskManager.Task::(prepare) Task=`aa006a8e-65f1-4d94-9269-831d134501c1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117361545563', 'lastCheck': '0.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2332::DEBUG::2012-12-09 12:56:32,569::task::588::TaskManager.Task::(_updateState) Task=`aa006a8e-65f1-4d94-9269-831d134501c1`::moving from state preparing -> state finished
>Thread-2332::DEBUG::2012-12-09 12:56:32,569::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2332::DEBUG::2012-12-09 12:56:32,570::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2332::DEBUG::2012-12-09 12:56:32,570::task::978::TaskManager.Task::(_decref) Task=`aa006a8e-65f1-4d94-9269-831d134501c1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:34,512::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:34,551::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163615 s, 62.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:56:34,924::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:34,925::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:35,027::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:35,028::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:35,028::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:35,029::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2335::DEBUG::2012-12-09 12:56:35,486::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2335::DEBUG::2012-12-09 12:56:35,486::task::588::TaskManager.Task::(_updateState) Task=`a64bbbf3-4b34-4c85-a802-1901c9216f88`::moving from state init -> state preparing
>Thread-2335::INFO::2012-12-09 12:56:35,486::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2335::INFO::2012-12-09 12:56:35,487::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2335::DEBUG::2012-12-09 12:56:35,487::task::1172::TaskManager.Task::(prepare) Task=`a64bbbf3-4b34-4c85-a802-1901c9216f88`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2335::DEBUG::2012-12-09 12:56:35,487::task::588::TaskManager.Task::(_updateState) Task=`a64bbbf3-4b34-4c85-a802-1901c9216f88`::moving from state preparing -> state finished
>Thread-2335::DEBUG::2012-12-09 12:56:35,487::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2335::DEBUG::2012-12-09 12:56:35,487::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2335::DEBUG::2012-12-09 12:56:35,488::task::978::TaskManager.Task::(_decref) Task=`a64bbbf3-4b34-4c85-a802-1901c9216f88`::ref 0 aborting False
>Thread-2336::DEBUG::2012-12-09 12:56:35,500::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2336::DEBUG::2012-12-09 12:56:35,500::task::588::TaskManager.Task::(_updateState) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::moving from state init -> state preparing
>Thread-2336::INFO::2012-12-09 12:56:35,500::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2336::DEBUG::2012-12-09 12:56:35,501::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c8f3307-1d6e-4e98-ac68-30a08c72974b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2336::DEBUG::2012-12-09 12:56:35,501::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2336::DEBUG::2012-12-09 12:56:35,501::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2336::DEBUG::2012-12-09 12:56:35,501::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9c8f3307-1d6e-4e98-ac68-30a08c72974b`::Granted request
>Thread-2336::DEBUG::2012-12-09 12:56:35,502::task::817::TaskManager.Task::(resourceAcquired) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2336::DEBUG::2012-12-09 12:56:35,502::task::978::TaskManager.Task::(_decref) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::ref 1 aborting False
>Thread-2336::DEBUG::2012-12-09 12:56:35,503::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2336::DEBUG::2012-12-09 12:56:35,503::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2336::DEBUG::2012-12-09 12:56:35,602::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2336::WARNING::2012-12-09 12:56:35,603::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2336::DEBUG::2012-12-09 12:56:35,604::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2336::WARNING::2012-12-09 12:56:35,604::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2336::INFO::2012-12-09 12:56:35,605::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2336::DEBUG::2012-12-09 12:56:35,605::task::1172::TaskManager.Task::(prepare) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2336::DEBUG::2012-12-09 12:56:35,605::task::588::TaskManager.Task::(_updateState) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::moving from state preparing -> state finished
>Thread-2336::DEBUG::2012-12-09 12:56:35,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2336::DEBUG::2012-12-09 12:56:35,605::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2336::DEBUG::2012-12-09 12:56:35,606::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2336::DEBUG::2012-12-09 12:56:35,606::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2336::DEBUG::2012-12-09 12:56:35,606::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2336::DEBUG::2012-12-09 12:56:35,606::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2336::DEBUG::2012-12-09 12:56:35,606::task::978::TaskManager.Task::(_decref) Task=`cbaa88a8-891e-4354-895d-8d91e04665ec`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:36,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:36,603::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188925 s, 54.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:38,610::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:38,651::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176459 s, 58.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:40,657::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:40,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170454 s, 60.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:56:41,870::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:41,881::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000445807 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:42,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:42,746::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164459 s, 62.3 MB/s\n'; <rc> = 0
>Thread-2340::DEBUG::2012-12-09 12:56:43,519::task::588::TaskManager.Task::(_updateState) Task=`a0a17e90-3532-498e-9cf9-c0d6254a22a6`::moving from state init -> state preparing
>Thread-2340::INFO::2012-12-09 12:56:43,519::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2340::INFO::2012-12-09 12:56:43,519::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011293888092', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-2340::DEBUG::2012-12-09 12:56:43,520::task::1172::TaskManager.Task::(prepare) Task=`a0a17e90-3532-498e-9cf9-c0d6254a22a6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011293888092', 'lastCheck': '1.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.5', 'code': 358, 'valid': False}}
>Thread-2340::DEBUG::2012-12-09 12:56:43,520::task::588::TaskManager.Task::(_updateState) Task=`a0a17e90-3532-498e-9cf9-c0d6254a22a6`::moving from state preparing -> state finished
>Thread-2340::DEBUG::2012-12-09 12:56:43,520::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2340::DEBUG::2012-12-09 12:56:43,521::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2340::DEBUG::2012-12-09 12:56:43,521::task::978::TaskManager.Task::(_decref) Task=`a0a17e90-3532-498e-9cf9-c0d6254a22a6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:44,752::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:44,792::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163391 s, 62.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:56:45,030::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:45,031::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:45,135::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:45,137::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:45,137::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:45,137::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2343::DEBUG::2012-12-09 12:56:46,682::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2343::DEBUG::2012-12-09 12:56:46,683::task::588::TaskManager.Task::(_updateState) Task=`430489ee-e1ea-48bf-849c-cfc3e79381f1`::moving from state init -> state preparing
>Thread-2343::INFO::2012-12-09 12:56:46,683::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2343::INFO::2012-12-09 12:56:46,683::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2343::DEBUG::2012-12-09 12:56:46,683::task::1172::TaskManager.Task::(prepare) Task=`430489ee-e1ea-48bf-849c-cfc3e79381f1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2343::DEBUG::2012-12-09 12:56:46,683::task::588::TaskManager.Task::(_updateState) Task=`430489ee-e1ea-48bf-849c-cfc3e79381f1`::moving from state preparing -> state finished
>Thread-2343::DEBUG::2012-12-09 12:56:46,684::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2343::DEBUG::2012-12-09 12:56:46,684::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2343::DEBUG::2012-12-09 12:56:46,684::task::978::TaskManager.Task::(_decref) Task=`430489ee-e1ea-48bf-849c-cfc3e79381f1`::ref 0 aborting False
>Thread-2344::DEBUG::2012-12-09 12:56:46,699::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2344::DEBUG::2012-12-09 12:56:46,699::task::588::TaskManager.Task::(_updateState) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::moving from state init -> state preparing
>Thread-2344::INFO::2012-12-09 12:56:46,700::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2344::DEBUG::2012-12-09 12:56:46,700::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`941a0960-136b-49a4-b0e9-82d39828583d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2344::DEBUG::2012-12-09 12:56:46,701::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2344::DEBUG::2012-12-09 12:56:46,701::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2344::DEBUG::2012-12-09 12:56:46,701::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`941a0960-136b-49a4-b0e9-82d39828583d`::Granted request
>Thread-2344::DEBUG::2012-12-09 12:56:46,702::task::817::TaskManager.Task::(resourceAcquired) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2344::DEBUG::2012-12-09 12:56:46,702::task::978::TaskManager.Task::(_decref) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::ref 1 aborting False
>Thread-2344::DEBUG::2012-12-09 12:56:46,703::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2344::DEBUG::2012-12-09 12:56:46,703::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2344::DEBUG::2012-12-09 12:56:46,801::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2344::WARNING::2012-12-09 12:56:46,802::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2344::DEBUG::2012-12-09 12:56:46,802::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2344::WARNING::2012-12-09 12:56:46,803::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2344::INFO::2012-12-09 12:56:46,803::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2344::DEBUG::2012-12-09 12:56:46,803::task::1172::TaskManager.Task::(prepare) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2344::DEBUG::2012-12-09 12:56:46,804::task::588::TaskManager.Task::(_updateState) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::moving from state preparing -> state finished
>Thread-2344::DEBUG::2012-12-09 12:56:46,804::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2344::DEBUG::2012-12-09 12:56:46,804::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2344::DEBUG::2012-12-09 12:56:46,804::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2344::DEBUG::2012-12-09 12:56:46,805::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2344::DEBUG::2012-12-09 12:56:46,805::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2344::DEBUG::2012-12-09 12:56:46,805::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2344::DEBUG::2012-12-09 12:56:46,805::task::978::TaskManager.Task::(_decref) Task=`638e2192-c0e8-4502-aed4-469aebb3b706`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:46,813::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:46,878::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277421 s, 36.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:48,888::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:48,930::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180484 s, 56.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:50,937::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:50,977::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164257 s, 62.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:56:51,884::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:56:51,894::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000384057 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:56:52,984::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:53,025::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167349 s, 61.2 MB/s\n'; <rc> = 0
>Thread-2348::DEBUG::2012-12-09 12:56:54,583::task::588::TaskManager.Task::(_updateState) Task=`766c525d-7bf5-45ea-b6b2-5cb86c7ec9e4`::moving from state init -> state preparing
>Thread-2348::INFO::2012-12-09 12:56:54,583::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2348::INFO::2012-12-09 12:56:54,583::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107460021973', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-2348::DEBUG::2012-12-09 12:56:54,584::task::1172::TaskManager.Task::(prepare) Task=`766c525d-7bf5-45ea-b6b2-5cb86c7ec9e4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107460021973', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-2348::DEBUG::2012-12-09 12:56:54,584::task::588::TaskManager.Task::(_updateState) Task=`766c525d-7bf5-45ea-b6b2-5cb86c7ec9e4`::moving from state preparing -> state finished
>Thread-2348::DEBUG::2012-12-09 12:56:54,584::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2348::DEBUG::2012-12-09 12:56:54,584::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2348::DEBUG::2012-12-09 12:56:54,584::task::978::TaskManager.Task::(_decref) Task=`766c525d-7bf5-45ea-b6b2-5cb86c7ec9e4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:55,032::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:55,075::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180249 s, 56.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:56:55,138::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:56:55,139::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:56:55,243::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:56:55,244::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:56:55,245::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:56:55,245::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:56:57,081::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:57,122::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155543 s, 65.8 MB/s\n'; <rc> = 0
>Thread-2351::DEBUG::2012-12-09 12:56:58,579::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2351::DEBUG::2012-12-09 12:56:58,580::task::588::TaskManager.Task::(_updateState) Task=`359ade81-ff4b-4fe7-ae5d-d064b7a8391b`::moving from state init -> state preparing
>Thread-2351::INFO::2012-12-09 12:56:58,580::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2351::INFO::2012-12-09 12:56:58,580::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2351::DEBUG::2012-12-09 12:56:58,581::task::1172::TaskManager.Task::(prepare) Task=`359ade81-ff4b-4fe7-ae5d-d064b7a8391b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2351::DEBUG::2012-12-09 12:56:58,581::task::588::TaskManager.Task::(_updateState) Task=`359ade81-ff4b-4fe7-ae5d-d064b7a8391b`::moving from state preparing -> state finished
>Thread-2351::DEBUG::2012-12-09 12:56:58,581::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2351::DEBUG::2012-12-09 12:56:58,581::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2351::DEBUG::2012-12-09 12:56:58,581::task::978::TaskManager.Task::(_decref) Task=`359ade81-ff4b-4fe7-ae5d-d064b7a8391b`::ref 0 aborting False
>Thread-2352::DEBUG::2012-12-09 12:56:58,595::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2352::DEBUG::2012-12-09 12:56:58,595::task::588::TaskManager.Task::(_updateState) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::moving from state init -> state preparing
>Thread-2352::INFO::2012-12-09 12:56:58,596::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2352::DEBUG::2012-12-09 12:56:58,596::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f2b8f7d-035d-4c7d-ab38-bcbcfb4c1d67`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2352::DEBUG::2012-12-09 12:56:58,596::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2352::DEBUG::2012-12-09 12:56:58,597::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2352::DEBUG::2012-12-09 12:56:58,597::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f2b8f7d-035d-4c7d-ab38-bcbcfb4c1d67`::Granted request
>Thread-2352::DEBUG::2012-12-09 12:56:58,597::task::817::TaskManager.Task::(resourceAcquired) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2352::DEBUG::2012-12-09 12:56:58,598::task::978::TaskManager.Task::(_decref) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::ref 1 aborting False
>Thread-2352::DEBUG::2012-12-09 12:56:58,599::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2352::DEBUG::2012-12-09 12:56:58,599::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2352::DEBUG::2012-12-09 12:56:58,702::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2352::WARNING::2012-12-09 12:56:58,703::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2352::DEBUG::2012-12-09 12:56:58,703::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2352::WARNING::2012-12-09 12:56:58,704::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2352::INFO::2012-12-09 12:56:58,704::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2352::DEBUG::2012-12-09 12:56:58,704::task::1172::TaskManager.Task::(prepare) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2352::DEBUG::2012-12-09 12:56:58,705::task::588::TaskManager.Task::(_updateState) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::moving from state preparing -> state finished
>Thread-2352::DEBUG::2012-12-09 12:56:58,705::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2352::DEBUG::2012-12-09 12:56:58,705::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2352::DEBUG::2012-12-09 12:56:58,705::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2352::DEBUG::2012-12-09 12:56:58,706::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2352::DEBUG::2012-12-09 12:56:58,706::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2352::DEBUG::2012-12-09 12:56:58,706::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2352::DEBUG::2012-12-09 12:56:58,706::task::978::TaskManager.Task::(_decref) Task=`77b0875c-034c-46d1-9aae-95fccaf3d5ee`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:56:59,129::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:56:59,169::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159402 s, 64.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:01,176::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:01,220::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187473 s, 54.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:57:01,897::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:01,909::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000501367 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:03,227::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:03,268::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171141 s, 59.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:57:05,246::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:05,247::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:05,275::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:05,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203814 s, 50.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:57:05,356::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:05,357::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:05,357::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:05,358::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2356::DEBUG::2012-12-09 12:57:05,376::task::588::TaskManager.Task::(_updateState) Task=`0489452f-3214-4796-8727-1b913ec5802c`::moving from state init -> state preparing
>Thread-2356::INFO::2012-12-09 12:57:05,377::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2356::INFO::2012-12-09 12:57:05,377::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125138759613', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-2356::DEBUG::2012-12-09 12:57:05,377::task::1172::TaskManager.Task::(prepare) Task=`0489452f-3214-4796-8727-1b913ec5802c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125138759613', 'lastCheck': '3.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-2356::DEBUG::2012-12-09 12:57:05,378::task::588::TaskManager.Task::(_updateState) Task=`0489452f-3214-4796-8727-1b913ec5802c`::moving from state preparing -> state finished
>Thread-2356::DEBUG::2012-12-09 12:57:05,378::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2356::DEBUG::2012-12-09 12:57:05,378::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2356::DEBUG::2012-12-09 12:57:05,379::task::978::TaskManager.Task::(_decref) Task=`0489452f-3214-4796-8727-1b913ec5802c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:07,329::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:07,369::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166507 s, 61.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:09,376::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:09,419::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189627 s, 54.0 MB/s\n'; <rc> = 0
>Thread-2359::DEBUG::2012-12-09 12:57:09,765::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2359::DEBUG::2012-12-09 12:57:09,766::task::588::TaskManager.Task::(_updateState) Task=`126819a8-3591-4b3e-9ad4-f40881cf5c86`::moving from state init -> state preparing
>Thread-2359::INFO::2012-12-09 12:57:09,766::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2359::INFO::2012-12-09 12:57:09,767::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2359::DEBUG::2012-12-09 12:57:09,767::task::1172::TaskManager.Task::(prepare) Task=`126819a8-3591-4b3e-9ad4-f40881cf5c86`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2359::DEBUG::2012-12-09 12:57:09,767::task::588::TaskManager.Task::(_updateState) Task=`126819a8-3591-4b3e-9ad4-f40881cf5c86`::moving from state preparing -> state finished
>Thread-2359::DEBUG::2012-12-09 12:57:09,767::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2359::DEBUG::2012-12-09 12:57:09,768::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2359::DEBUG::2012-12-09 12:57:09,768::task::978::TaskManager.Task::(_decref) Task=`126819a8-3591-4b3e-9ad4-f40881cf5c86`::ref 0 aborting False
>Thread-2360::DEBUG::2012-12-09 12:57:09,781::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2360::DEBUG::2012-12-09 12:57:09,781::task::588::TaskManager.Task::(_updateState) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::moving from state init -> state preparing
>Thread-2360::INFO::2012-12-09 12:57:09,782::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2360::DEBUG::2012-12-09 12:57:09,782::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`87ad60a9-d06b-4f40-9948-55ec7caef52a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2360::DEBUG::2012-12-09 12:57:09,782::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2360::DEBUG::2012-12-09 12:57:09,783::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2360::DEBUG::2012-12-09 12:57:09,783::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`87ad60a9-d06b-4f40-9948-55ec7caef52a`::Granted request
>Thread-2360::DEBUG::2012-12-09 12:57:09,783::task::817::TaskManager.Task::(resourceAcquired) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2360::DEBUG::2012-12-09 12:57:09,783::task::978::TaskManager.Task::(_decref) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::ref 1 aborting False
>Thread-2360::DEBUG::2012-12-09 12:57:09,784::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2360::DEBUG::2012-12-09 12:57:09,785::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2360::DEBUG::2012-12-09 12:57:09,882::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2360::WARNING::2012-12-09 12:57:09,884::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2360::DEBUG::2012-12-09 12:57:09,884::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2360::WARNING::2012-12-09 12:57:09,885::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2360::INFO::2012-12-09 12:57:09,886::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2360::DEBUG::2012-12-09 12:57:09,886::task::1172::TaskManager.Task::(prepare) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2360::DEBUG::2012-12-09 12:57:09,887::task::588::TaskManager.Task::(_updateState) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::moving from state preparing -> state finished
>Thread-2360::DEBUG::2012-12-09 12:57:09,887::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2360::DEBUG::2012-12-09 12:57:09,888::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2360::DEBUG::2012-12-09 12:57:09,888::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2360::DEBUG::2012-12-09 12:57:09,889::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2360::DEBUG::2012-12-09 12:57:09,889::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2360::DEBUG::2012-12-09 12:57:09,889::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2360::DEBUG::2012-12-09 12:57:09,890::task::978::TaskManager.Task::(_decref) Task=`06ec8404-36b7-4e0b-ab33-f7d93cf0d1d0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:11,427::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:11,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016378 s, 62.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:57:11,912::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:11,923::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000451261 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:13,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:13,517::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185326 s, 55.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:57:15,359::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:15,359::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:57:15,453::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:15,454::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:15,454::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:15,455::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:57:15,524::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:15,561::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160432 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:17,568::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:17,610::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176117 s, 58.1 MB/s\n'; <rc> = 0
>Thread-2364::DEBUG::2012-12-09 12:57:19,260::task::588::TaskManager.Task::(_updateState) Task=`050160a5-675e-47e0-b44b-7a43022e13e3`::moving from state init -> state preparing
>Thread-2364::INFO::2012-12-09 12:57:19,261::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2364::INFO::2012-12-09 12:57:19,261::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114240646362', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-2364::DEBUG::2012-12-09 12:57:19,261::task::1172::TaskManager.Task::(prepare) Task=`050160a5-675e-47e0-b44b-7a43022e13e3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114240646362', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-2364::DEBUG::2012-12-09 12:57:19,262::task::588::TaskManager.Task::(_updateState) Task=`050160a5-675e-47e0-b44b-7a43022e13e3`::moving from state preparing -> state finished
>Thread-2364::DEBUG::2012-12-09 12:57:19,262::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2364::DEBUG::2012-12-09 12:57:19,262::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2364::DEBUG::2012-12-09 12:57:19,263::task::978::TaskManager.Task::(_decref) Task=`050160a5-675e-47e0-b44b-7a43022e13e3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:19,617::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:19,654::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159337 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:21,660::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:21,697::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159216 s, 64.3 MB/s\n'; <rc> = 0
>Thread-2367::DEBUG::2012-12-09 12:57:21,808::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2367::DEBUG::2012-12-09 12:57:21,808::task::588::TaskManager.Task::(_updateState) Task=`81cd158d-9473-45c8-9156-2bef8b7255b7`::moving from state init -> state preparing
>Thread-2367::INFO::2012-12-09 12:57:21,809::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2367::INFO::2012-12-09 12:57:21,809::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2367::DEBUG::2012-12-09 12:57:21,809::task::1172::TaskManager.Task::(prepare) Task=`81cd158d-9473-45c8-9156-2bef8b7255b7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2367::DEBUG::2012-12-09 12:57:21,809::task::588::TaskManager.Task::(_updateState) Task=`81cd158d-9473-45c8-9156-2bef8b7255b7`::moving from state preparing -> state finished
>Thread-2367::DEBUG::2012-12-09 12:57:21,809::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2367::DEBUG::2012-12-09 12:57:21,810::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2367::DEBUG::2012-12-09 12:57:21,810::task::978::TaskManager.Task::(_decref) Task=`81cd158d-9473-45c8-9156-2bef8b7255b7`::ref 0 aborting False
>Thread-2368::DEBUG::2012-12-09 12:57:21,821::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2368::DEBUG::2012-12-09 12:57:21,822::task::588::TaskManager.Task::(_updateState) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::moving from state init -> state preparing
>Thread-2368::INFO::2012-12-09 12:57:21,822::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2368::DEBUG::2012-12-09 12:57:21,822::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7869932a-136f-4927-b6da-b3c77a191e2d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2368::DEBUG::2012-12-09 12:57:21,823::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2368::DEBUG::2012-12-09 12:57:21,823::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2368::DEBUG::2012-12-09 12:57:21,823::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7869932a-136f-4927-b6da-b3c77a191e2d`::Granted request
>Thread-2368::DEBUG::2012-12-09 12:57:21,823::task::817::TaskManager.Task::(resourceAcquired) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2368::DEBUG::2012-12-09 12:57:21,824::task::978::TaskManager.Task::(_decref) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::ref 1 aborting False
>Thread-2368::DEBUG::2012-12-09 12:57:21,824::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2368::DEBUG::2012-12-09 12:57:21,825::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2368::DEBUG::2012-12-09 12:57:21,918::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2368::WARNING::2012-12-09 12:57:21,919::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2368::DEBUG::2012-12-09 12:57:21,919::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2368::WARNING::2012-12-09 12:57:21,920::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2368::INFO::2012-12-09 12:57:21,920::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2368::DEBUG::2012-12-09 12:57:21,920::task::1172::TaskManager.Task::(prepare) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2368::DEBUG::2012-12-09 12:57:21,921::task::588::TaskManager.Task::(_updateState) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::moving from state preparing -> state finished
>Thread-2368::DEBUG::2012-12-09 12:57:21,921::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2368::DEBUG::2012-12-09 12:57:21,921::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2368::DEBUG::2012-12-09 12:57:21,921::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2368::DEBUG::2012-12-09 12:57:21,922::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2368::DEBUG::2012-12-09 12:57:21,922::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2368::DEBUG::2012-12-09 12:57:21,922::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2368::DEBUG::2012-12-09 12:57:21,922::task::978::TaskManager.Task::(_decref) Task=`89e9776c-45c8-4700-bf8e-49a227e5a34b`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:57:21,926::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:21,934::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000427394 s, 9.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:23,704::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:23,742::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015732 s, 65.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:57:25,456::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:25,457::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:57:25,560::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:25,561::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:25,562::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:25,562::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:57:25,749::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:25,796::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023507 s, 43.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:27,803::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:27,842::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163139 s, 62.8 MB/s\n'; <rc> = 0
>Thread-2372::DEBUG::2012-12-09 12:57:29,718::task::588::TaskManager.Task::(_updateState) Task=`aa0b3f03-5337-4fe8-84b7-70bb3c74706a`::moving from state init -> state preparing
>Thread-2372::INFO::2012-12-09 12:57:29,719::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2372::INFO::2012-12-09 12:57:29,720::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00825500488281', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2372::DEBUG::2012-12-09 12:57:29,720::task::1172::TaskManager.Task::(prepare) Task=`aa0b3f03-5337-4fe8-84b7-70bb3c74706a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00825500488281', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2372::DEBUG::2012-12-09 12:57:29,720::task::588::TaskManager.Task::(_updateState) Task=`aa0b3f03-5337-4fe8-84b7-70bb3c74706a`::moving from state preparing -> state finished
>Thread-2372::DEBUG::2012-12-09 12:57:29,721::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2372::DEBUG::2012-12-09 12:57:29,721::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2372::DEBUG::2012-12-09 12:57:29,721::task::978::TaskManager.Task::(_decref) Task=`aa0b3f03-5337-4fe8-84b7-70bb3c74706a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:29,849::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:29,888::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160502 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:31,895::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:31,937::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:31,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000532604 s, 7.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:31,960::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216896 s, 47.2 MB/s\n'; <rc> = 0
>Thread-2375::DEBUG::2012-12-09 12:57:33,488::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2375::DEBUG::2012-12-09 12:57:33,488::task::588::TaskManager.Task::(_updateState) Task=`86dc0eb0-b080-41da-b035-364a22f6134e`::moving from state init -> state preparing
>Thread-2375::INFO::2012-12-09 12:57:33,488::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2375::INFO::2012-12-09 12:57:33,489::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2375::DEBUG::2012-12-09 12:57:33,489::task::1172::TaskManager.Task::(prepare) Task=`86dc0eb0-b080-41da-b035-364a22f6134e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2375::DEBUG::2012-12-09 12:57:33,489::task::588::TaskManager.Task::(_updateState) Task=`86dc0eb0-b080-41da-b035-364a22f6134e`::moving from state preparing -> state finished
>Thread-2375::DEBUG::2012-12-09 12:57:33,489::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2375::DEBUG::2012-12-09 12:57:33,489::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2375::DEBUG::2012-12-09 12:57:33,490::task::978::TaskManager.Task::(_decref) Task=`86dc0eb0-b080-41da-b035-364a22f6134e`::ref 0 aborting False
>Thread-2376::DEBUG::2012-12-09 12:57:33,505::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2376::DEBUG::2012-12-09 12:57:33,506::task::588::TaskManager.Task::(_updateState) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::moving from state init -> state preparing
>Thread-2376::INFO::2012-12-09 12:57:33,507::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2376::DEBUG::2012-12-09 12:57:33,508::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff850dbd-c063-4135-9ca1-7bdcf36915cb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2376::DEBUG::2012-12-09 12:57:33,508::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2376::DEBUG::2012-12-09 12:57:33,509::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2376::DEBUG::2012-12-09 12:57:33,509::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff850dbd-c063-4135-9ca1-7bdcf36915cb`::Granted request
>Thread-2376::DEBUG::2012-12-09 12:57:33,510::task::817::TaskManager.Task::(resourceAcquired) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2376::DEBUG::2012-12-09 12:57:33,510::task::978::TaskManager.Task::(_decref) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::ref 1 aborting False
>Thread-2376::DEBUG::2012-12-09 12:57:33,511::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2376::DEBUG::2012-12-09 12:57:33,511::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2376::DEBUG::2012-12-09 12:57:33,613::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2376::WARNING::2012-12-09 12:57:33,615::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2376::DEBUG::2012-12-09 12:57:33,615::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2376::WARNING::2012-12-09 12:57:33,615::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2376::INFO::2012-12-09 12:57:33,616::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2376::DEBUG::2012-12-09 12:57:33,616::task::1172::TaskManager.Task::(prepare) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2376::DEBUG::2012-12-09 12:57:33,616::task::588::TaskManager.Task::(_updateState) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::moving from state preparing -> state finished
>Thread-2376::DEBUG::2012-12-09 12:57:33,617::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2376::DEBUG::2012-12-09 12:57:33,617::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2376::DEBUG::2012-12-09 12:57:33,617::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2376::DEBUG::2012-12-09 12:57:33,617::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2376::DEBUG::2012-12-09 12:57:33,617::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2376::DEBUG::2012-12-09 12:57:33,618::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2376::DEBUG::2012-12-09 12:57:33,618::task::978::TaskManager.Task::(_decref) Task=`1941a147-2399-42eb-a892-8ea737b78a41`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:33,967::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:34,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188339 s, 54.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:57:35,563::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:35,564::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:57:35,670::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:35,672::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:35,672::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:35,672::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:57:36,015::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:36,053::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161423 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:38,060::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:38,108::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240151 s, 42.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:40,115::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:40,156::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181902 s, 56.3 MB/s\n'; <rc> = 0
>Thread-2380::DEBUG::2012-12-09 12:57:41,387::task::588::TaskManager.Task::(_updateState) Task=`d1db5009-1f06-4dc1-b74f-1871bd1c811f`::moving from state init -> state preparing
>Thread-2380::INFO::2012-12-09 12:57:41,387::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2380::INFO::2012-12-09 12:57:41,387::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136060714722', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-2380::DEBUG::2012-12-09 12:57:41,387::task::1172::TaskManager.Task::(prepare) Task=`d1db5009-1f06-4dc1-b74f-1871bd1c811f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136060714722', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-2380::DEBUG::2012-12-09 12:57:41,388::task::588::TaskManager.Task::(_updateState) Task=`d1db5009-1f06-4dc1-b74f-1871bd1c811f`::moving from state preparing -> state finished
>Thread-2380::DEBUG::2012-12-09 12:57:41,388::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2380::DEBUG::2012-12-09 12:57:41,388::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2380::DEBUG::2012-12-09 12:57:41,388::task::978::TaskManager.Task::(_decref) Task=`d1db5009-1f06-4dc1-b74f-1871bd1c811f`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:57:41,956::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:41,967::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000413316 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:42,163::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:42,202::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163117 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:44,209::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:44,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219527 s, 46.6 MB/s\n'; <rc> = 0
>Thread-2383::DEBUG::2012-12-09 12:57:44,643::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2383::DEBUG::2012-12-09 12:57:44,644::task::588::TaskManager.Task::(_updateState) Task=`40bf3551-c5f4-4186-a274-2ca1f514f25f`::moving from state init -> state preparing
>Thread-2383::INFO::2012-12-09 12:57:44,644::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2383::INFO::2012-12-09 12:57:44,644::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2383::DEBUG::2012-12-09 12:57:44,645::task::1172::TaskManager.Task::(prepare) Task=`40bf3551-c5f4-4186-a274-2ca1f514f25f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2383::DEBUG::2012-12-09 12:57:44,645::task::588::TaskManager.Task::(_updateState) Task=`40bf3551-c5f4-4186-a274-2ca1f514f25f`::moving from state preparing -> state finished
>Thread-2383::DEBUG::2012-12-09 12:57:44,645::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2383::DEBUG::2012-12-09 12:57:44,645::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2383::DEBUG::2012-12-09 12:57:44,645::task::978::TaskManager.Task::(_decref) Task=`40bf3551-c5f4-4186-a274-2ca1f514f25f`::ref 0 aborting False
>Thread-2384::DEBUG::2012-12-09 12:57:44,659::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2384::DEBUG::2012-12-09 12:57:44,659::task::588::TaskManager.Task::(_updateState) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::moving from state init -> state preparing
>Thread-2384::INFO::2012-12-09 12:57:44,660::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2384::DEBUG::2012-12-09 12:57:44,660::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e0d92e07-57c6-4e51-82dd-0581abc9c448`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2384::DEBUG::2012-12-09 12:57:44,660::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2384::DEBUG::2012-12-09 12:57:44,660::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2384::DEBUG::2012-12-09 12:57:44,661::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e0d92e07-57c6-4e51-82dd-0581abc9c448`::Granted request
>Thread-2384::DEBUG::2012-12-09 12:57:44,661::task::817::TaskManager.Task::(resourceAcquired) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2384::DEBUG::2012-12-09 12:57:44,661::task::978::TaskManager.Task::(_decref) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::ref 1 aborting False
>Thread-2384::DEBUG::2012-12-09 12:57:44,662::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2384::DEBUG::2012-12-09 12:57:44,662::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2384::DEBUG::2012-12-09 12:57:44,760::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2384::WARNING::2012-12-09 12:57:44,761::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2384::DEBUG::2012-12-09 12:57:44,762::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2384::WARNING::2012-12-09 12:57:44,762::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2384::INFO::2012-12-09 12:57:44,762::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2384::DEBUG::2012-12-09 12:57:44,763::task::1172::TaskManager.Task::(prepare) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2384::DEBUG::2012-12-09 12:57:44,763::task::588::TaskManager.Task::(_updateState) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::moving from state preparing -> state finished
>Thread-2384::DEBUG::2012-12-09 12:57:44,763::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2384::DEBUG::2012-12-09 12:57:44,763::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2384::DEBUG::2012-12-09 12:57:44,764::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2384::DEBUG::2012-12-09 12:57:44,764::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2384::DEBUG::2012-12-09 12:57:44,764::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2384::DEBUG::2012-12-09 12:57:44,764::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2384::DEBUG::2012-12-09 12:57:44,764::task::978::TaskManager.Task::(_decref) Task=`5b253c34-fab3-47bf-8dee-e56fa7bb80ea`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:57:45,673::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:45,674::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:57:45,790::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:45,791::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:45,792::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:45,792::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:57:46,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:46,309::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181537 s, 56.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:48,317::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:48,391::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338609 s, 30.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:50,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:50,438::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171334 s, 59.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:57:51,970::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:57:51,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000718385 s, 5.7 MB/s\n'; <rc> = 0
>Thread-2388::DEBUG::2012-12-09 12:57:52,390::task::588::TaskManager.Task::(_updateState) Task=`218f4f0d-e348-47fc-85b1-1127167c689a`::moving from state init -> state preparing
>Thread-2388::INFO::2012-12-09 12:57:52,391::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2388::INFO::2012-12-09 12:57:52,391::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00883293151855', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2388::DEBUG::2012-12-09 12:57:52,391::task::1172::TaskManager.Task::(prepare) Task=`218f4f0d-e348-47fc-85b1-1127167c689a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00883293151855', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-2388::DEBUG::2012-12-09 12:57:52,391::task::588::TaskManager.Task::(_updateState) Task=`218f4f0d-e348-47fc-85b1-1127167c689a`::moving from state preparing -> state finished
>Thread-2388::DEBUG::2012-12-09 12:57:52,392::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2388::DEBUG::2012-12-09 12:57:52,392::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2388::DEBUG::2012-12-09 12:57:52,392::task::978::TaskManager.Task::(_decref) Task=`218f4f0d-e348-47fc-85b1-1127167c689a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:57:52,445::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:52,489::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182121 s, 56.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:54,496::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:54,536::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174218 s, 58.8 MB/s\n'; <rc> = 0
>Thread-2390::DEBUG::2012-12-09 12:57:55,485::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2390::DEBUG::2012-12-09 12:57:55,485::task::588::TaskManager.Task::(_updateState) Task=`6117bf4a-efa3-4ee8-a33c-b93aa330941b`::moving from state init -> state preparing
>Thread-2390::INFO::2012-12-09 12:57:55,486::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2390::INFO::2012-12-09 12:57:55,486::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2390::DEBUG::2012-12-09 12:57:55,486::task::1172::TaskManager.Task::(prepare) Task=`6117bf4a-efa3-4ee8-a33c-b93aa330941b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2390::DEBUG::2012-12-09 12:57:55,486::task::588::TaskManager.Task::(_updateState) Task=`6117bf4a-efa3-4ee8-a33c-b93aa330941b`::moving from state preparing -> state finished
>Thread-2390::DEBUG::2012-12-09 12:57:55,487::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2390::DEBUG::2012-12-09 12:57:55,487::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2390::DEBUG::2012-12-09 12:57:55,487::task::978::TaskManager.Task::(_decref) Task=`6117bf4a-efa3-4ee8-a33c-b93aa330941b`::ref 0 aborting False
>Thread-2391::DEBUG::2012-12-09 12:57:55,499::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2391::DEBUG::2012-12-09 12:57:55,500::task::588::TaskManager.Task::(_updateState) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::moving from state init -> state preparing
>Thread-2391::INFO::2012-12-09 12:57:55,500::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2391::DEBUG::2012-12-09 12:57:55,500::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`69386cca-666b-44d7-9e62-d8f243ad1e57`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2391::DEBUG::2012-12-09 12:57:55,500::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2391::DEBUG::2012-12-09 12:57:55,501::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2391::DEBUG::2012-12-09 12:57:55,501::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`69386cca-666b-44d7-9e62-d8f243ad1e57`::Granted request
>Thread-2391::DEBUG::2012-12-09 12:57:55,501::task::817::TaskManager.Task::(resourceAcquired) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2391::DEBUG::2012-12-09 12:57:55,501::task::978::TaskManager.Task::(_decref) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::ref 1 aborting False
>Thread-2391::DEBUG::2012-12-09 12:57:55,502::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2391::DEBUG::2012-12-09 12:57:55,502::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2391::DEBUG::2012-12-09 12:57:55,602::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2391::WARNING::2012-12-09 12:57:55,603::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2391::DEBUG::2012-12-09 12:57:55,603::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2391::WARNING::2012-12-09 12:57:55,604::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2391::INFO::2012-12-09 12:57:55,604::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2391::DEBUG::2012-12-09 12:57:55,604::task::1172::TaskManager.Task::(prepare) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2391::DEBUG::2012-12-09 12:57:55,604::task::588::TaskManager.Task::(_updateState) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::moving from state preparing -> state finished
>Thread-2391::DEBUG::2012-12-09 12:57:55,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2391::DEBUG::2012-12-09 12:57:55,605::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2391::DEBUG::2012-12-09 12:57:55,605::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2391::DEBUG::2012-12-09 12:57:55,605::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2391::DEBUG::2012-12-09 12:57:55,606::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2391::DEBUG::2012-12-09 12:57:55,606::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2391::DEBUG::2012-12-09 12:57:55,606::task::978::TaskManager.Task::(_decref) Task=`77e9421c-6cbf-4ef7-9613-15c17240f94b`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:57:55,793::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:57:55,794::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:57:55,903::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:57:55,904::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:57:55,904::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:57:55,905::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:57:56,542::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:56,583::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163372 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:57:58,589::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:57:58,629::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166067 s, 61.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:00,636::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:00,674::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160352 s, 63.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:01,981::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:01,992::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000592868 s, 6.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:02,681::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:02,722::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157934 s, 64.8 MB/s\n'; <rc> = 0
>Thread-2396::DEBUG::2012-12-09 12:58:03,771::task::588::TaskManager.Task::(_updateState) Task=`60bbeea1-9b01-4e8e-bb65-885a3cb7c78b`::moving from state init -> state preparing
>Thread-2396::INFO::2012-12-09 12:58:03,772::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2396::INFO::2012-12-09 12:58:03,772::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011155128479', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-2396::DEBUG::2012-12-09 12:58:03,773::task::1172::TaskManager.Task::(prepare) Task=`60bbeea1-9b01-4e8e-bb65-885a3cb7c78b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.011155128479', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-2396::DEBUG::2012-12-09 12:58:03,773::task::588::TaskManager.Task::(_updateState) Task=`60bbeea1-9b01-4e8e-bb65-885a3cb7c78b`::moving from state preparing -> state finished
>Thread-2396::DEBUG::2012-12-09 12:58:03,774::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2396::DEBUG::2012-12-09 12:58:03,774::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2396::DEBUG::2012-12-09 12:58:03,775::task::978::TaskManager.Task::(_decref) Task=`60bbeea1-9b01-4e8e-bb65-885a3cb7c78b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:04,728::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:04,775::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023738 s, 43.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:58:05,906::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:05,906::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:06,004::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:06,005::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:06,005::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:06,006::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:06,782::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:06,840::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234581 s, 43.7 MB/s\n'; <rc> = 0
>Thread-2399::DEBUG::2012-12-09 12:58:07,958::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2399::DEBUG::2012-12-09 12:58:07,959::task::588::TaskManager.Task::(_updateState) Task=`af9710f2-f980-4821-8d94-95cad463faf2`::moving from state init -> state preparing
>Thread-2399::INFO::2012-12-09 12:58:07,959::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2399::INFO::2012-12-09 12:58:07,960::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2399::DEBUG::2012-12-09 12:58:07,960::task::1172::TaskManager.Task::(prepare) Task=`af9710f2-f980-4821-8d94-95cad463faf2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2399::DEBUG::2012-12-09 12:58:07,960::task::588::TaskManager.Task::(_updateState) Task=`af9710f2-f980-4821-8d94-95cad463faf2`::moving from state preparing -> state finished
>Thread-2399::DEBUG::2012-12-09 12:58:07,960::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2399::DEBUG::2012-12-09 12:58:07,961::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2399::DEBUG::2012-12-09 12:58:07,961::task::978::TaskManager.Task::(_decref) Task=`af9710f2-f980-4821-8d94-95cad463faf2`::ref 0 aborting False
>Thread-2400::DEBUG::2012-12-09 12:58:07,973::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2400::DEBUG::2012-12-09 12:58:07,974::task::588::TaskManager.Task::(_updateState) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::moving from state init -> state preparing
>Thread-2400::INFO::2012-12-09 12:58:07,974::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2400::DEBUG::2012-12-09 12:58:07,974::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cfc404d6-4a09-4410-ae02-8026a9731ba3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2400::DEBUG::2012-12-09 12:58:07,974::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2400::DEBUG::2012-12-09 12:58:07,975::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2400::DEBUG::2012-12-09 12:58:07,975::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cfc404d6-4a09-4410-ae02-8026a9731ba3`::Granted request
>Thread-2400::DEBUG::2012-12-09 12:58:07,975::task::817::TaskManager.Task::(resourceAcquired) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2400::DEBUG::2012-12-09 12:58:07,975::task::978::TaskManager.Task::(_decref) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::ref 1 aborting False
>Thread-2400::DEBUG::2012-12-09 12:58:07,976::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2400::DEBUG::2012-12-09 12:58:07,977::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2400::DEBUG::2012-12-09 12:58:08,075::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2400::WARNING::2012-12-09 12:58:08,076::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2400::DEBUG::2012-12-09 12:58:08,076::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2400::WARNING::2012-12-09 12:58:08,077::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2400::INFO::2012-12-09 12:58:08,077::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2400::DEBUG::2012-12-09 12:58:08,077::task::1172::TaskManager.Task::(prepare) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2400::DEBUG::2012-12-09 12:58:08,078::task::588::TaskManager.Task::(_updateState) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::moving from state preparing -> state finished
>Thread-2400::DEBUG::2012-12-09 12:58:08,078::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2400::DEBUG::2012-12-09 12:58:08,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2400::DEBUG::2012-12-09 12:58:08,078::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2400::DEBUG::2012-12-09 12:58:08,079::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2400::DEBUG::2012-12-09 12:58:08,079::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2400::DEBUG::2012-12-09 12:58:08,079::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2400::DEBUG::2012-12-09 12:58:08,079::task::978::TaskManager.Task::(_decref) Task=`e91efe89-f9cc-4591-95a2-3fbbef204562`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:08,848::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:08,914::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261923 s, 39.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:10,921::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:10,964::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187668 s, 54.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:11,995::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:12,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000443451 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:12,972::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:13,012::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165638 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:15,019::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:15,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176764 s, 57.9 MB/s\n'; <rc> = 0
>Thread-2404::DEBUG::2012-12-09 12:58:15,566::task::588::TaskManager.Task::(_updateState) Task=`59a3cffe-5e3e-4fc6-96a6-47fe736659e2`::moving from state init -> state preparing
>Thread-2404::INFO::2012-12-09 12:58:15,566::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2404::INFO::2012-12-09 12:58:15,566::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140941143036', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-2404::DEBUG::2012-12-09 12:58:15,567::task::1172::TaskManager.Task::(prepare) Task=`59a3cffe-5e3e-4fc6-96a6-47fe736659e2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140941143036', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-2404::DEBUG::2012-12-09 12:58:15,567::task::588::TaskManager.Task::(_updateState) Task=`59a3cffe-5e3e-4fc6-96a6-47fe736659e2`::moving from state preparing -> state finished
>Thread-2404::DEBUG::2012-12-09 12:58:15,567::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2404::DEBUG::2012-12-09 12:58:15,567::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2404::DEBUG::2012-12-09 12:58:15,567::task::978::TaskManager.Task::(_decref) Task=`59a3cffe-5e3e-4fc6-96a6-47fe736659e2`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:58:16,006::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:16,007::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:16,109::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:16,110::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:16,110::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:16,111::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:17,068::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:17,108::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164046 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:19,115::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:19,156::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174957 s, 58.5 MB/s\n'; <rc> = 0
>Thread-2407::DEBUG::2012-12-09 12:58:20,179::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2407::DEBUG::2012-12-09 12:58:20,180::task::588::TaskManager.Task::(_updateState) Task=`57b205b9-ea34-460f-a0cd-5a99790766f5`::moving from state init -> state preparing
>Thread-2407::INFO::2012-12-09 12:58:20,180::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2407::INFO::2012-12-09 12:58:20,180::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2407::DEBUG::2012-12-09 12:58:20,181::task::1172::TaskManager.Task::(prepare) Task=`57b205b9-ea34-460f-a0cd-5a99790766f5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2407::DEBUG::2012-12-09 12:58:20,181::task::588::TaskManager.Task::(_updateState) Task=`57b205b9-ea34-460f-a0cd-5a99790766f5`::moving from state preparing -> state finished
>Thread-2407::DEBUG::2012-12-09 12:58:20,181::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2407::DEBUG::2012-12-09 12:58:20,181::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2407::DEBUG::2012-12-09 12:58:20,181::task::978::TaskManager.Task::(_decref) Task=`57b205b9-ea34-460f-a0cd-5a99790766f5`::ref 0 aborting False
>Thread-2408::DEBUG::2012-12-09 12:58:20,196::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2408::DEBUG::2012-12-09 12:58:20,196::task::588::TaskManager.Task::(_updateState) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::moving from state init -> state preparing
>Thread-2408::INFO::2012-12-09 12:58:20,197::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2408::DEBUG::2012-12-09 12:58:20,197::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a88b0eda-8dc7-4eef-b48b-e0c6a0996940`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2408::DEBUG::2012-12-09 12:58:20,197::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2408::DEBUG::2012-12-09 12:58:20,198::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2408::DEBUG::2012-12-09 12:58:20,198::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a88b0eda-8dc7-4eef-b48b-e0c6a0996940`::Granted request
>Thread-2408::DEBUG::2012-12-09 12:58:20,198::task::817::TaskManager.Task::(resourceAcquired) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2408::DEBUG::2012-12-09 12:58:20,198::task::978::TaskManager.Task::(_decref) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::ref 1 aborting False
>Thread-2408::DEBUG::2012-12-09 12:58:20,199::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2408::DEBUG::2012-12-09 12:58:20,200::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2408::DEBUG::2012-12-09 12:58:20,303::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2408::WARNING::2012-12-09 12:58:20,304::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2408::DEBUG::2012-12-09 12:58:20,304::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2408::WARNING::2012-12-09 12:58:20,305::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2408::INFO::2012-12-09 12:58:20,305::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2408::DEBUG::2012-12-09 12:58:20,305::task::1172::TaskManager.Task::(prepare) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2408::DEBUG::2012-12-09 12:58:20,305::task::588::TaskManager.Task::(_updateState) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::moving from state preparing -> state finished
>Thread-2408::DEBUG::2012-12-09 12:58:20,306::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2408::DEBUG::2012-12-09 12:58:20,306::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2408::DEBUG::2012-12-09 12:58:20,306::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2408::DEBUG::2012-12-09 12:58:20,306::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2408::DEBUG::2012-12-09 12:58:20,307::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2408::DEBUG::2012-12-09 12:58:20,307::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2408::DEBUG::2012-12-09 12:58:20,307::task::978::TaskManager.Task::(_decref) Task=`c9da6309-d5bd-4636-b900-ab2dfd729abc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:21,164::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:21,203::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161814 s, 63.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:22,013::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:22,021::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000445064 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:23,210::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:23,250::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167273 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:25,257::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:25,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246349 s, 41.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:58:26,112::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:26,112::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:26,217::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:26,219::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:26,219::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:26,219::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:27,322::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-2412::DEBUG::2012-12-09 12:58:27,330::task::588::TaskManager.Task::(_updateState) Task=`e497fd80-8400-4fa8-9658-57c9669d9080`::moving from state init -> state preparing
>Thread-2412::INFO::2012-12-09 12:58:27,332::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2412::INFO::2012-12-09 12:58:27,332::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00844788551331', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2412::DEBUG::2012-12-09 12:58:27,333::task::1172::TaskManager.Task::(prepare) Task=`e497fd80-8400-4fa8-9658-57c9669d9080`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00844788551331', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2412::DEBUG::2012-12-09 12:58:27,334::task::588::TaskManager.Task::(_updateState) Task=`e497fd80-8400-4fa8-9658-57c9669d9080`::moving from state preparing -> state finished
>Thread-2412::DEBUG::2012-12-09 12:58:27,336::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2412::DEBUG::2012-12-09 12:58:27,337::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2412::DEBUG::2012-12-09 12:58:27,338::task::978::TaskManager.Task::(_decref) Task=`e497fd80-8400-4fa8-9658-57c9669d9080`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:27,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0359028 s, 28.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:29,392::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:29,437::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203136 s, 50.4 MB/s\n'; <rc> = 0
>Thread-2414::DEBUG::2012-12-09 12:58:31,196::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2414::DEBUG::2012-12-09 12:58:31,197::task::588::TaskManager.Task::(_updateState) Task=`02bc462d-44a3-4473-b7ad-ff073c4c33f3`::moving from state init -> state preparing
>Thread-2414::INFO::2012-12-09 12:58:31,197::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2414::INFO::2012-12-09 12:58:31,198::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2414::DEBUG::2012-12-09 12:58:31,198::task::1172::TaskManager.Task::(prepare) Task=`02bc462d-44a3-4473-b7ad-ff073c4c33f3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2414::DEBUG::2012-12-09 12:58:31,198::task::588::TaskManager.Task::(_updateState) Task=`02bc462d-44a3-4473-b7ad-ff073c4c33f3`::moving from state preparing -> state finished
>Thread-2414::DEBUG::2012-12-09 12:58:31,199::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2414::DEBUG::2012-12-09 12:58:31,199::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2414::DEBUG::2012-12-09 12:58:31,199::task::978::TaskManager.Task::(_decref) Task=`02bc462d-44a3-4473-b7ad-ff073c4c33f3`::ref 0 aborting False
>Thread-2415::DEBUG::2012-12-09 12:58:31,212::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2415::DEBUG::2012-12-09 12:58:31,213::task::588::TaskManager.Task::(_updateState) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::moving from state init -> state preparing
>Thread-2415::INFO::2012-12-09 12:58:31,213::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2415::DEBUG::2012-12-09 12:58:31,214::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b961db6d-c0ad-4c65-a148-b962dea265e1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2415::DEBUG::2012-12-09 12:58:31,214::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2415::DEBUG::2012-12-09 12:58:31,214::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2415::DEBUG::2012-12-09 12:58:31,214::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b961db6d-c0ad-4c65-a148-b962dea265e1`::Granted request
>Thread-2415::DEBUG::2012-12-09 12:58:31,215::task::817::TaskManager.Task::(resourceAcquired) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2415::DEBUG::2012-12-09 12:58:31,215::task::978::TaskManager.Task::(_decref) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::ref 1 aborting False
>Thread-2415::DEBUG::2012-12-09 12:58:31,215::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2415::DEBUG::2012-12-09 12:58:31,216::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2415::DEBUG::2012-12-09 12:58:31,313::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2415::WARNING::2012-12-09 12:58:31,314::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2415::DEBUG::2012-12-09 12:58:31,314::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2415::WARNING::2012-12-09 12:58:31,315::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2415::INFO::2012-12-09 12:58:31,315::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2415::DEBUG::2012-12-09 12:58:31,315::task::1172::TaskManager.Task::(prepare) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2415::DEBUG::2012-12-09 12:58:31,315::task::588::TaskManager.Task::(_updateState) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::moving from state preparing -> state finished
>Thread-2415::DEBUG::2012-12-09 12:58:31,316::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2415::DEBUG::2012-12-09 12:58:31,316::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2415::DEBUG::2012-12-09 12:58:31,316::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2415::DEBUG::2012-12-09 12:58:31,316::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2415::DEBUG::2012-12-09 12:58:31,317::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2415::DEBUG::2012-12-09 12:58:31,317::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2415::DEBUG::2012-12-09 12:58:31,317::task::978::TaskManager.Task::(_decref) Task=`39676b68-f6ab-47f5-9d2c-eab4e11d2155`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:31,443::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:31,483::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168729 s, 60.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:32,024::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:32,035::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000365376 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:33,490::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:33,529::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154119 s, 66.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:35,536::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:35,584::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248829 s, 41.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:58:36,220::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:36,221::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:36,317::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:36,318::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:36,319::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:36,319::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:37,590::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:37,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161067 s, 63.6 MB/s\n'; <rc> = 0
>Thread-2420::DEBUG::2012-12-09 12:58:39,531::task::588::TaskManager.Task::(_updateState) Task=`126a285d-6df8-4dcc-a6e4-1868134c66a4`::moving from state init -> state preparing
>Thread-2420::INFO::2012-12-09 12:58:39,532::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2420::INFO::2012-12-09 12:58:39,532::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118441581726', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-2420::DEBUG::2012-12-09 12:58:39,532::task::1172::TaskManager.Task::(prepare) Task=`126a285d-6df8-4dcc-a6e4-1868134c66a4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118441581726', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.2', 'code': 358, 'valid': False}}
>Thread-2420::DEBUG::2012-12-09 12:58:39,533::task::588::TaskManager.Task::(_updateState) Task=`126a285d-6df8-4dcc-a6e4-1868134c66a4`::moving from state preparing -> state finished
>Thread-2420::DEBUG::2012-12-09 12:58:39,533::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2420::DEBUG::2012-12-09 12:58:39,533::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2420::DEBUG::2012-12-09 12:58:39,533::task::978::TaskManager.Task::(_decref) Task=`126a285d-6df8-4dcc-a6e4-1868134c66a4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:39,635::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:39,674::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160378 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:41,681::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:41,724::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019678 s, 52.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:42,038::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:42,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000498203 s, 8.2 MB/s\n'; <rc> = 0
>Thread-2422::DEBUG::2012-12-09 12:58:42,381::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2422::DEBUG::2012-12-09 12:58:42,381::task::588::TaskManager.Task::(_updateState) Task=`997a2edf-3e2c-4535-8da8-cb417ef7fa50`::moving from state init -> state preparing
>Thread-2422::INFO::2012-12-09 12:58:42,382::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2422::INFO::2012-12-09 12:58:42,382::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2422::DEBUG::2012-12-09 12:58:42,382::task::1172::TaskManager.Task::(prepare) Task=`997a2edf-3e2c-4535-8da8-cb417ef7fa50`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2422::DEBUG::2012-12-09 12:58:42,382::task::588::TaskManager.Task::(_updateState) Task=`997a2edf-3e2c-4535-8da8-cb417ef7fa50`::moving from state preparing -> state finished
>Thread-2422::DEBUG::2012-12-09 12:58:42,383::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2422::DEBUG::2012-12-09 12:58:42,383::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2422::DEBUG::2012-12-09 12:58:42,383::task::978::TaskManager.Task::(_decref) Task=`997a2edf-3e2c-4535-8da8-cb417ef7fa50`::ref 0 aborting False
>Thread-2423::DEBUG::2012-12-09 12:58:42,395::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2423::DEBUG::2012-12-09 12:58:42,396::task::588::TaskManager.Task::(_updateState) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::moving from state init -> state preparing
>Thread-2423::INFO::2012-12-09 12:58:42,397::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2423::DEBUG::2012-12-09 12:58:42,397::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0624006b-e910-4550-a662-33b513605536`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2423::DEBUG::2012-12-09 12:58:42,398::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2423::DEBUG::2012-12-09 12:58:42,398::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2423::DEBUG::2012-12-09 12:58:42,398::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0624006b-e910-4550-a662-33b513605536`::Granted request
>Thread-2423::DEBUG::2012-12-09 12:58:42,399::task::817::TaskManager.Task::(resourceAcquired) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2423::DEBUG::2012-12-09 12:58:42,399::task::978::TaskManager.Task::(_decref) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::ref 1 aborting False
>Thread-2423::DEBUG::2012-12-09 12:58:42,400::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2423::DEBUG::2012-12-09 12:58:42,401::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2423::DEBUG::2012-12-09 12:58:42,502::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2423::WARNING::2012-12-09 12:58:42,503::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2423::DEBUG::2012-12-09 12:58:42,503::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2423::WARNING::2012-12-09 12:58:42,504::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2423::INFO::2012-12-09 12:58:42,504::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2423::DEBUG::2012-12-09 12:58:42,505::task::1172::TaskManager.Task::(prepare) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2423::DEBUG::2012-12-09 12:58:42,505::task::588::TaskManager.Task::(_updateState) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::moving from state preparing -> state finished
>Thread-2423::DEBUG::2012-12-09 12:58:42,505::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2423::DEBUG::2012-12-09 12:58:42,505::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2423::DEBUG::2012-12-09 12:58:42,506::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2423::DEBUG::2012-12-09 12:58:42,506::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2423::DEBUG::2012-12-09 12:58:42,506::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2423::DEBUG::2012-12-09 12:58:42,506::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2423::DEBUG::2012-12-09 12:58:42,506::task::978::TaskManager.Task::(_decref) Task=`06b03d65-1da0-41d0-bb78-7a1c1b75b6e9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:43,731::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:43,771::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158296 s, 64.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:45,777::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:45,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158958 s, 64.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:58:46,320::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:46,321::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:46,423::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:46,424::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:46,425::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:46,425::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:47,822::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:47,879::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161921 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:58:49,889::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:49,931::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192012 s, 53.3 MB/s\n'; <rc> = 0
>Thread-2428::DEBUG::2012-12-09 12:58:50,821::task::588::TaskManager.Task::(_updateState) Task=`17929907-9bf7-4087-a68b-41ad4a2e0189`::moving from state init -> state preparing
>Thread-2428::INFO::2012-12-09 12:58:50,822::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2428::INFO::2012-12-09 12:58:50,823::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119800567627', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-2428::DEBUG::2012-12-09 12:58:50,823::task::1172::TaskManager.Task::(prepare) Task=`17929907-9bf7-4087-a68b-41ad4a2e0189`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119800567627', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-2428::DEBUG::2012-12-09 12:58:50,824::task::588::TaskManager.Task::(_updateState) Task=`17929907-9bf7-4087-a68b-41ad4a2e0189`::moving from state preparing -> state finished
>Thread-2428::DEBUG::2012-12-09 12:58:50,824::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2428::DEBUG::2012-12-09 12:58:50,825::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2428::DEBUG::2012-12-09 12:58:50,825::task::978::TaskManager.Task::(_decref) Task=`17929907-9bf7-4087-a68b-41ad4a2e0189`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:51,938::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:51,982::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188671 s, 54.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:58:52,053::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:58:52,064::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000485531 s, 8.4 MB/s\n'; <rc> = 0
>Thread-2431::DEBUG::2012-12-09 12:58:53,919::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2431::DEBUG::2012-12-09 12:58:53,920::task::588::TaskManager.Task::(_updateState) Task=`333e2aa8-945d-43d1-b0f6-1aef2f3559d3`::moving from state init -> state preparing
>Thread-2431::INFO::2012-12-09 12:58:53,921::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2431::INFO::2012-12-09 12:58:53,921::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2431::DEBUG::2012-12-09 12:58:53,921::task::1172::TaskManager.Task::(prepare) Task=`333e2aa8-945d-43d1-b0f6-1aef2f3559d3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2431::DEBUG::2012-12-09 12:58:53,921::task::588::TaskManager.Task::(_updateState) Task=`333e2aa8-945d-43d1-b0f6-1aef2f3559d3`::moving from state preparing -> state finished
>Thread-2431::DEBUG::2012-12-09 12:58:53,922::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2431::DEBUG::2012-12-09 12:58:53,922::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2431::DEBUG::2012-12-09 12:58:53,922::task::978::TaskManager.Task::(_decref) Task=`333e2aa8-945d-43d1-b0f6-1aef2f3559d3`::ref 0 aborting False
>Thread-2432::DEBUG::2012-12-09 12:58:53,935::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2432::DEBUG::2012-12-09 12:58:53,936::task::588::TaskManager.Task::(_updateState) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::moving from state init -> state preparing
>Thread-2432::INFO::2012-12-09 12:58:53,936::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2432::DEBUG::2012-12-09 12:58:53,936::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f9bf14a-800b-420a-9449-6a0499bd5e81`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2432::DEBUG::2012-12-09 12:58:53,937::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2432::DEBUG::2012-12-09 12:58:53,937::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2432::DEBUG::2012-12-09 12:58:53,937::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f9bf14a-800b-420a-9449-6a0499bd5e81`::Granted request
>Thread-2432::DEBUG::2012-12-09 12:58:53,937::task::817::TaskManager.Task::(resourceAcquired) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2432::DEBUG::2012-12-09 12:58:53,938::task::978::TaskManager.Task::(_decref) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::ref 1 aborting False
>Thread-2432::DEBUG::2012-12-09 12:58:53,938::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2432::DEBUG::2012-12-09 12:58:53,939::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:53,989::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:54,033::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207821 s, 49.3 MB/s\n'; <rc> = 0
>Thread-2432::DEBUG::2012-12-09 12:58:54,047::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2432::WARNING::2012-12-09 12:58:54,048::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2432::DEBUG::2012-12-09 12:58:54,049::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2432::WARNING::2012-12-09 12:58:54,049::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2432::INFO::2012-12-09 12:58:54,049::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2432::DEBUG::2012-12-09 12:58:54,050::task::1172::TaskManager.Task::(prepare) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2432::DEBUG::2012-12-09 12:58:54,050::task::588::TaskManager.Task::(_updateState) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::moving from state preparing -> state finished
>Thread-2432::DEBUG::2012-12-09 12:58:54,050::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2432::DEBUG::2012-12-09 12:58:54,050::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2432::DEBUG::2012-12-09 12:58:54,051::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2432::DEBUG::2012-12-09 12:58:54,051::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2432::DEBUG::2012-12-09 12:58:54,051::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2432::DEBUG::2012-12-09 12:58:54,051::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2432::DEBUG::2012-12-09 12:58:54,052::task::978::TaskManager.Task::(_decref) Task=`722ba662-f7c4-4d4a-9b28-193748af8dd7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:58:56,040::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:56,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174521 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:58:56,426::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:58:56,426::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:58:56,534::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:58:56,535::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:58:56,535::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:58:56,536::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:58:58,087::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:58:58,132::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203832 s, 50.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:00,139::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:00,181::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172616 s, 59.3 MB/s\n'; <rc> = 0
>Thread-2436::DEBUG::2012-12-09 12:59:01,515::task::588::TaskManager.Task::(_updateState) Task=`a76bd15e-9c81-43f3-9342-7e9f9c1b9e21`::moving from state init -> state preparing
>Thread-2436::INFO::2012-12-09 12:59:01,516::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2436::INFO::2012-12-09 12:59:01,516::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116832256317', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-2436::DEBUG::2012-12-09 12:59:01,517::task::1172::TaskManager.Task::(prepare) Task=`a76bd15e-9c81-43f3-9342-7e9f9c1b9e21`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116832256317', 'lastCheck': '9.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-2436::DEBUG::2012-12-09 12:59:01,517::task::588::TaskManager.Task::(_updateState) Task=`a76bd15e-9c81-43f3-9342-7e9f9c1b9e21`::moving from state preparing -> state finished
>Thread-2436::DEBUG::2012-12-09 12:59:01,517::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2436::DEBUG::2012-12-09 12:59:01,517::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2436::DEBUG::2012-12-09 12:59:01,517::task::978::TaskManager.Task::(_decref) Task=`a76bd15e-9c81-43f3-9342-7e9f9c1b9e21`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 12:59:02,067::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:02,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000418609 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:02,187::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:02,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161628 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:04,233::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:04,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172248 s, 59.4 MB/s\n'; <rc> = 0
>Thread-2439::DEBUG::2012-12-09 12:59:05,983::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2439::DEBUG::2012-12-09 12:59:05,984::task::588::TaskManager.Task::(_updateState) Task=`484a3e35-0c5d-4cdf-9443-3ad7ea65feea`::moving from state init -> state preparing
>Thread-2439::INFO::2012-12-09 12:59:05,984::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2439::INFO::2012-12-09 12:59:05,984::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2439::DEBUG::2012-12-09 12:59:05,984::task::1172::TaskManager.Task::(prepare) Task=`484a3e35-0c5d-4cdf-9443-3ad7ea65feea`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2439::DEBUG::2012-12-09 12:59:05,984::task::588::TaskManager.Task::(_updateState) Task=`484a3e35-0c5d-4cdf-9443-3ad7ea65feea`::moving from state preparing -> state finished
>Thread-2439::DEBUG::2012-12-09 12:59:05,985::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2439::DEBUG::2012-12-09 12:59:05,985::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2439::DEBUG::2012-12-09 12:59:05,985::task::978::TaskManager.Task::(_decref) Task=`484a3e35-0c5d-4cdf-9443-3ad7ea65feea`::ref 0 aborting False
>Thread-2440::DEBUG::2012-12-09 12:59:05,998::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2440::DEBUG::2012-12-09 12:59:05,998::task::588::TaskManager.Task::(_updateState) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::moving from state init -> state preparing
>Thread-2440::INFO::2012-12-09 12:59:05,999::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2440::DEBUG::2012-12-09 12:59:05,999::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e441ffd3-a247-47d8-85c0-257f1dfd1148`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2440::DEBUG::2012-12-09 12:59:05,999::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2440::DEBUG::2012-12-09 12:59:05,999::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2440::DEBUG::2012-12-09 12:59:06,000::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e441ffd3-a247-47d8-85c0-257f1dfd1148`::Granted request
>Thread-2440::DEBUG::2012-12-09 12:59:06,000::task::817::TaskManager.Task::(resourceAcquired) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2440::DEBUG::2012-12-09 12:59:06,000::task::978::TaskManager.Task::(_decref) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::ref 1 aborting False
>Thread-2440::DEBUG::2012-12-09 12:59:06,001::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2440::DEBUG::2012-12-09 12:59:06,001::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2440::DEBUG::2012-12-09 12:59:06,106::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2440::WARNING::2012-12-09 12:59:06,107::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2440::DEBUG::2012-12-09 12:59:06,107::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2440::WARNING::2012-12-09 12:59:06,108::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2440::INFO::2012-12-09 12:59:06,108::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2440::DEBUG::2012-12-09 12:59:06,108::task::1172::TaskManager.Task::(prepare) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2440::DEBUG::2012-12-09 12:59:06,108::task::588::TaskManager.Task::(_updateState) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::moving from state preparing -> state finished
>Thread-2440::DEBUG::2012-12-09 12:59:06,109::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2440::DEBUG::2012-12-09 12:59:06,109::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2440::DEBUG::2012-12-09 12:59:06,109::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2440::DEBUG::2012-12-09 12:59:06,109::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2440::DEBUG::2012-12-09 12:59:06,110::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2440::DEBUG::2012-12-09 12:59:06,110::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2440::DEBUG::2012-12-09 12:59:06,110::task::978::TaskManager.Task::(_decref) Task=`13da72de-528e-45d3-90ea-5bfc46c6cbe1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:06,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:06,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177946 s, 57.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:59:06,537::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:06,537::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:59:06,637::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:06,638::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:06,638::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:06,639::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:59:08,328::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:08,381::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207255 s, 49.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:10,388::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:10,430::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182229 s, 56.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:12,080::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 12:59:12,081::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:59:12,082::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:12,092::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00105223 s, 1.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:12,093::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00105223 s, 1.9 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 12:59:12,093::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 12:59:12,093::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 12:59:12,093::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 12:59:12,094::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,094::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,094::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,094::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,095::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,095::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:12,192::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:12,194::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 12:59:12,194::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 12:59:12,195::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 12:59:12,195::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 12:59:12,195::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 12:59:12,195::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:12,295::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:12,296::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:12,305::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000692805 s, 5.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:12,436::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:12,475::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166772 s, 61.4 MB/s\n'; <rc> = 0
>Thread-2444::DEBUG::2012-12-09 12:59:12,910::task::588::TaskManager.Task::(_updateState) Task=`97071af9-0802-475e-8b03-5b36b980e069`::moving from state init -> state preparing
>Thread-2444::INFO::2012-12-09 12:59:12,911::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2444::INFO::2012-12-09 12:59:12,912::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.010312795639', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.3', 'code': 358, 'valid': False}}
>Thread-2444::DEBUG::2012-12-09 12:59:12,912::task::1172::TaskManager.Task::(prepare) Task=`97071af9-0802-475e-8b03-5b36b980e069`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.010312795639', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.3', 'code': 358, 'valid': False}}
>Thread-2444::DEBUG::2012-12-09 12:59:12,913::task::588::TaskManager.Task::(_updateState) Task=`97071af9-0802-475e-8b03-5b36b980e069`::moving from state preparing -> state finished
>Thread-2444::DEBUG::2012-12-09 12:59:12,913::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2444::DEBUG::2012-12-09 12:59:12,913::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2444::DEBUG::2012-12-09 12:59:12,914::task::978::TaskManager.Task::(_decref) Task=`97071af9-0802-475e-8b03-5b36b980e069`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:14,482::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:14,524::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187019 s, 54.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:16,531::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:16,578::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215528 s, 47.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:59:16,640::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:16,641::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:59:16,757::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:16,758::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:16,758::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:16,759::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2447::DEBUG::2012-12-09 12:59:17,201::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2447::DEBUG::2012-12-09 12:59:17,202::task::588::TaskManager.Task::(_updateState) Task=`8b5c9819-b1cb-44fc-8a2b-7db7d4e1d6ec`::moving from state init -> state preparing
>Thread-2447::INFO::2012-12-09 12:59:17,202::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2447::INFO::2012-12-09 12:59:17,203::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2447::DEBUG::2012-12-09 12:59:17,203::task::1172::TaskManager.Task::(prepare) Task=`8b5c9819-b1cb-44fc-8a2b-7db7d4e1d6ec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2447::DEBUG::2012-12-09 12:59:17,204::task::588::TaskManager.Task::(_updateState) Task=`8b5c9819-b1cb-44fc-8a2b-7db7d4e1d6ec`::moving from state preparing -> state finished
>Thread-2447::DEBUG::2012-12-09 12:59:17,204::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2447::DEBUG::2012-12-09 12:59:17,204::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2447::DEBUG::2012-12-09 12:59:17,205::task::978::TaskManager.Task::(_decref) Task=`8b5c9819-b1cb-44fc-8a2b-7db7d4e1d6ec`::ref 0 aborting False
>Thread-2448::DEBUG::2012-12-09 12:59:17,221::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2448::DEBUG::2012-12-09 12:59:17,222::task::588::TaskManager.Task::(_updateState) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::moving from state init -> state preparing
>Thread-2448::INFO::2012-12-09 12:59:17,222::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2448::DEBUG::2012-12-09 12:59:17,222::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1d59387-c9cf-4b79-bd8e-f96656a6b1e9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2448::DEBUG::2012-12-09 12:59:17,223::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2448::DEBUG::2012-12-09 12:59:17,223::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2448::DEBUG::2012-12-09 12:59:17,223::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c1d59387-c9cf-4b79-bd8e-f96656a6b1e9`::Granted request
>Thread-2448::DEBUG::2012-12-09 12:59:17,223::task::817::TaskManager.Task::(resourceAcquired) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2448::DEBUG::2012-12-09 12:59:17,223::task::978::TaskManager.Task::(_decref) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::ref 1 aborting False
>Thread-2448::DEBUG::2012-12-09 12:59:17,224::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2448::DEBUG::2012-12-09 12:59:17,225::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2448::DEBUG::2012-12-09 12:59:17,335::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2448::WARNING::2012-12-09 12:59:17,337::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2448::DEBUG::2012-12-09 12:59:17,338::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2448::WARNING::2012-12-09 12:59:17,339::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2448::INFO::2012-12-09 12:59:17,340::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2448::DEBUG::2012-12-09 12:59:17,340::task::1172::TaskManager.Task::(prepare) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2448::DEBUG::2012-12-09 12:59:17,341::task::588::TaskManager.Task::(_updateState) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::moving from state preparing -> state finished
>Thread-2448::DEBUG::2012-12-09 12:59:17,341::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2448::DEBUG::2012-12-09 12:59:17,342::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2448::DEBUG::2012-12-09 12:59:17,342::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2448::DEBUG::2012-12-09 12:59:17,343::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2448::DEBUG::2012-12-09 12:59:17,343::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2448::DEBUG::2012-12-09 12:59:17,344::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2448::DEBUG::2012-12-09 12:59:17,344::task::978::TaskManager.Task::(_decref) Task=`680a1c3d-e412-42c9-9a1d-3d7daac426d9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:18,585::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:18,627::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017256 s, 59.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:20,635::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:20,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165477 s, 61.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:22,310::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:22,323::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000831555 s, 4.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:22,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:22,732::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239755 s, 42.7 MB/s\n'; <rc> = 0
>Thread-2452::DEBUG::2012-12-09 12:59:24,071::task::588::TaskManager.Task::(_updateState) Task=`4513d93f-a09f-47ca-bbd4-9e598eda53d6`::moving from state init -> state preparing
>Thread-2452::INFO::2012-12-09 12:59:24,072::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2452::INFO::2012-12-09 12:59:24,072::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0142049789429', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-2452::DEBUG::2012-12-09 12:59:24,073::task::1172::TaskManager.Task::(prepare) Task=`4513d93f-a09f-47ca-bbd4-9e598eda53d6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0142049789429', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-2452::DEBUG::2012-12-09 12:59:24,073::task::588::TaskManager.Task::(_updateState) Task=`4513d93f-a09f-47ca-bbd4-9e598eda53d6`::moving from state preparing -> state finished
>Thread-2452::DEBUG::2012-12-09 12:59:24,073::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2452::DEBUG::2012-12-09 12:59:24,074::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2452::DEBUG::2012-12-09 12:59:24,074::task::978::TaskManager.Task::(_decref) Task=`4513d93f-a09f-47ca-bbd4-9e598eda53d6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:24,739::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:24,786::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206856 s, 49.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:59:26,760::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:26,761::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:26,793::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:26,836::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167419 s, 61.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:59:26,869::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:26,870::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:26,871::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:26,871::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2455::DEBUG::2012-12-09 12:59:28,001::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2455::DEBUG::2012-12-09 12:59:28,003::task::588::TaskManager.Task::(_updateState) Task=`2442e90a-ac02-4e7f-bfde-4b39ad5e0029`::moving from state init -> state preparing
>Thread-2455::INFO::2012-12-09 12:59:28,003::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2455::INFO::2012-12-09 12:59:28,004::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2455::DEBUG::2012-12-09 12:59:28,004::task::1172::TaskManager.Task::(prepare) Task=`2442e90a-ac02-4e7f-bfde-4b39ad5e0029`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2455::DEBUG::2012-12-09 12:59:28,005::task::588::TaskManager.Task::(_updateState) Task=`2442e90a-ac02-4e7f-bfde-4b39ad5e0029`::moving from state preparing -> state finished
>Thread-2455::DEBUG::2012-12-09 12:59:28,005::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2455::DEBUG::2012-12-09 12:59:28,006::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2455::DEBUG::2012-12-09 12:59:28,006::task::978::TaskManager.Task::(_decref) Task=`2442e90a-ac02-4e7f-bfde-4b39ad5e0029`::ref 0 aborting False
>Thread-2456::DEBUG::2012-12-09 12:59:28,021::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2456::DEBUG::2012-12-09 12:59:28,022::task::588::TaskManager.Task::(_updateState) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::moving from state init -> state preparing
>Thread-2456::INFO::2012-12-09 12:59:28,023::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2456::DEBUG::2012-12-09 12:59:28,023::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`771232e7-f70f-41af-a98b-8974a18f3323`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2456::DEBUG::2012-12-09 12:59:28,024::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2456::DEBUG::2012-12-09 12:59:28,025::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2456::DEBUG::2012-12-09 12:59:28,025::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`771232e7-f70f-41af-a98b-8974a18f3323`::Granted request
>Thread-2456::DEBUG::2012-12-09 12:59:28,026::task::817::TaskManager.Task::(resourceAcquired) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2456::DEBUG::2012-12-09 12:59:28,026::task::978::TaskManager.Task::(_decref) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::ref 1 aborting False
>Thread-2456::DEBUG::2012-12-09 12:59:28,027::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2456::DEBUG::2012-12-09 12:59:28,027::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2456::DEBUG::2012-12-09 12:59:28,126::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2456::WARNING::2012-12-09 12:59:28,127::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2456::DEBUG::2012-12-09 12:59:28,127::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2456::WARNING::2012-12-09 12:59:28,128::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2456::INFO::2012-12-09 12:59:28,128::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2456::DEBUG::2012-12-09 12:59:28,128::task::1172::TaskManager.Task::(prepare) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2456::DEBUG::2012-12-09 12:59:28,129::task::588::TaskManager.Task::(_updateState) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::moving from state preparing -> state finished
>Thread-2456::DEBUG::2012-12-09 12:59:28,129::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2456::DEBUG::2012-12-09 12:59:28,129::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2456::DEBUG::2012-12-09 12:59:28,129::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2456::DEBUG::2012-12-09 12:59:28,129::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2456::DEBUG::2012-12-09 12:59:28,130::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2456::DEBUG::2012-12-09 12:59:28,130::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2456::DEBUG::2012-12-09 12:59:28,130::task::978::TaskManager.Task::(_decref) Task=`e7247ab2-8209-47a8-b38e-c5e381aef8f2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:28,843::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:28,891::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0232404 s, 44.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:30,898::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:30,938::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166499 s, 61.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:32,329::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:32,340::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000376989 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:32,945::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:32,987::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173175 s, 59.1 MB/s\n'; <rc> = 0
>Thread-2460::DEBUG::2012-12-09 12:59:34,477::task::588::TaskManager.Task::(_updateState) Task=`3759a0a4-e9b3-4a7a-be13-90cedfae648f`::moving from state init -> state preparing
>Thread-2460::INFO::2012-12-09 12:59:34,478::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2460::INFO::2012-12-09 12:59:34,478::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0115509033203', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2460::DEBUG::2012-12-09 12:59:34,479::task::1172::TaskManager.Task::(prepare) Task=`3759a0a4-e9b3-4a7a-be13-90cedfae648f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0115509033203', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2460::DEBUG::2012-12-09 12:59:34,479::task::588::TaskManager.Task::(_updateState) Task=`3759a0a4-e9b3-4a7a-be13-90cedfae648f`::moving from state preparing -> state finished
>Thread-2460::DEBUG::2012-12-09 12:59:34,479::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2460::DEBUG::2012-12-09 12:59:34,480::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2460::DEBUG::2012-12-09 12:59:34,480::task::978::TaskManager.Task::(_decref) Task=`3759a0a4-e9b3-4a7a-be13-90cedfae648f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:34,994::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:35,034::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017772 s, 57.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 12:59:36,872::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 12:59:36,872::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:36,873::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:59:36,971::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:36,972::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:36,972::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:36,973::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:59:37,041::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:37,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162305 s, 63.1 MB/s\n'; <rc> = 0
>Thread-2463::DEBUG::2012-12-09 12:59:38,565::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2463::DEBUG::2012-12-09 12:59:38,566::task::588::TaskManager.Task::(_updateState) Task=`1b731eab-9e57-4281-beac-d478b6f5b11d`::moving from state init -> state preparing
>Thread-2463::INFO::2012-12-09 12:59:38,566::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2463::INFO::2012-12-09 12:59:38,566::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2463::DEBUG::2012-12-09 12:59:38,566::task::1172::TaskManager.Task::(prepare) Task=`1b731eab-9e57-4281-beac-d478b6f5b11d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2463::DEBUG::2012-12-09 12:59:38,567::task::588::TaskManager.Task::(_updateState) Task=`1b731eab-9e57-4281-beac-d478b6f5b11d`::moving from state preparing -> state finished
>Thread-2463::DEBUG::2012-12-09 12:59:38,567::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2463::DEBUG::2012-12-09 12:59:38,567::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2463::DEBUG::2012-12-09 12:59:38,567::task::978::TaskManager.Task::(_decref) Task=`1b731eab-9e57-4281-beac-d478b6f5b11d`::ref 0 aborting False
>Thread-2464::DEBUG::2012-12-09 12:59:38,581::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2464::DEBUG::2012-12-09 12:59:38,582::task::588::TaskManager.Task::(_updateState) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::moving from state init -> state preparing
>Thread-2464::INFO::2012-12-09 12:59:38,582::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2464::DEBUG::2012-12-09 12:59:38,583::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c088e6c7-396d-4a7d-a350-4a9b701fd2c5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2464::DEBUG::2012-12-09 12:59:38,583::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2464::DEBUG::2012-12-09 12:59:38,583::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2464::DEBUG::2012-12-09 12:59:38,584::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c088e6c7-396d-4a7d-a350-4a9b701fd2c5`::Granted request
>Thread-2464::DEBUG::2012-12-09 12:59:38,584::task::817::TaskManager.Task::(resourceAcquired) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2464::DEBUG::2012-12-09 12:59:38,584::task::978::TaskManager.Task::(_decref) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::ref 1 aborting False
>Thread-2464::DEBUG::2012-12-09 12:59:38,585::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2464::DEBUG::2012-12-09 12:59:38,586::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2464::DEBUG::2012-12-09 12:59:38,683::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2464::WARNING::2012-12-09 12:59:38,684::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2464::DEBUG::2012-12-09 12:59:38,684::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2464::WARNING::2012-12-09 12:59:38,685::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2464::INFO::2012-12-09 12:59:38,685::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2464::DEBUG::2012-12-09 12:59:38,685::task::1172::TaskManager.Task::(prepare) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2464::DEBUG::2012-12-09 12:59:38,686::task::588::TaskManager.Task::(_updateState) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::moving from state preparing -> state finished
>Thread-2464::DEBUG::2012-12-09 12:59:38,686::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2464::DEBUG::2012-12-09 12:59:38,686::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2464::DEBUG::2012-12-09 12:59:38,686::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2464::DEBUG::2012-12-09 12:59:38,687::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2464::DEBUG::2012-12-09 12:59:38,687::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2464::DEBUG::2012-12-09 12:59:38,687::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2464::DEBUG::2012-12-09 12:59:38,687::task::978::TaskManager.Task::(_decref) Task=`2eff7f5c-4d70-48d9-be41-f871cd913211`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:39,087::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:39,132::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214313 s, 47.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:41,138::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:41,180::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172363 s, 59.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:42,344::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:42,357::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00079637 s, 5.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:43,187::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:43,231::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019517 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:45,238::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:45,289::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261454 s, 39.2 MB/s\n'; <rc> = 0
>Thread-2468::DEBUG::2012-12-09 12:59:46,324::task::588::TaskManager.Task::(_updateState) Task=`3accc579-4884-4c66-980f-71cbd096c7db`::moving from state init -> state preparing
>Thread-2468::INFO::2012-12-09 12:59:46,324::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2468::INFO::2012-12-09 12:59:46,325::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136740207672', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-2468::DEBUG::2012-12-09 12:59:46,325::task::1172::TaskManager.Task::(prepare) Task=`3accc579-4884-4c66-980f-71cbd096c7db`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136740207672', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.4', 'code': 358, 'valid': False}}
>Thread-2468::DEBUG::2012-12-09 12:59:46,325::task::588::TaskManager.Task::(_updateState) Task=`3accc579-4884-4c66-980f-71cbd096c7db`::moving from state preparing -> state finished
>Thread-2468::DEBUG::2012-12-09 12:59:46,325::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2468::DEBUG::2012-12-09 12:59:46,325::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2468::DEBUG::2012-12-09 12:59:46,326::task::978::TaskManager.Task::(_decref) Task=`3accc579-4884-4c66-980f-71cbd096c7db`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:59:46,974::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:46,974::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:59:47,073::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:47,074::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:47,075::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:47,075::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:59:47,296::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:47,338::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170183 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:49,345::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:49,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166011 s, 61.7 MB/s\n'; <rc> = 0
>Thread-2471::DEBUG::2012-12-09 12:59:50,083::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2471::DEBUG::2012-12-09 12:59:50,084::task::588::TaskManager.Task::(_updateState) Task=`2ea6a5ff-3aa4-4ffc-a62d-37099f761744`::moving from state init -> state preparing
>Thread-2471::INFO::2012-12-09 12:59:50,084::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2471::INFO::2012-12-09 12:59:50,084::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2471::DEBUG::2012-12-09 12:59:50,084::task::1172::TaskManager.Task::(prepare) Task=`2ea6a5ff-3aa4-4ffc-a62d-37099f761744`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2471::DEBUG::2012-12-09 12:59:50,085::task::588::TaskManager.Task::(_updateState) Task=`2ea6a5ff-3aa4-4ffc-a62d-37099f761744`::moving from state preparing -> state finished
>Thread-2471::DEBUG::2012-12-09 12:59:50,085::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2471::DEBUG::2012-12-09 12:59:50,085::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2471::DEBUG::2012-12-09 12:59:50,085::task::978::TaskManager.Task::(_decref) Task=`2ea6a5ff-3aa4-4ffc-a62d-37099f761744`::ref 0 aborting False
>Thread-2472::DEBUG::2012-12-09 12:59:50,100::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2472::DEBUG::2012-12-09 12:59:50,101::task::588::TaskManager.Task::(_updateState) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::moving from state init -> state preparing
>Thread-2472::INFO::2012-12-09 12:59:50,101::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2472::DEBUG::2012-12-09 12:59:50,102::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fcd7d9f5-3dcf-46dc-99e9-2aaa843ad2ec`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2472::DEBUG::2012-12-09 12:59:50,102::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2472::DEBUG::2012-12-09 12:59:50,103::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2472::DEBUG::2012-12-09 12:59:50,103::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fcd7d9f5-3dcf-46dc-99e9-2aaa843ad2ec`::Granted request
>Thread-2472::DEBUG::2012-12-09 12:59:50,104::task::817::TaskManager.Task::(resourceAcquired) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2472::DEBUG::2012-12-09 12:59:50,104::task::978::TaskManager.Task::(_decref) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::ref 1 aborting False
>Thread-2472::DEBUG::2012-12-09 12:59:50,105::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2472::DEBUG::2012-12-09 12:59:50,106::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2472::DEBUG::2012-12-09 12:59:50,208::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2472::WARNING::2012-12-09 12:59:50,209::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2472::DEBUG::2012-12-09 12:59:50,209::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2472::WARNING::2012-12-09 12:59:50,210::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2472::INFO::2012-12-09 12:59:50,210::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2472::DEBUG::2012-12-09 12:59:50,210::task::1172::TaskManager.Task::(prepare) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2472::DEBUG::2012-12-09 12:59:50,210::task::588::TaskManager.Task::(_updateState) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::moving from state preparing -> state finished
>Thread-2472::DEBUG::2012-12-09 12:59:50,211::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2472::DEBUG::2012-12-09 12:59:50,211::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2472::DEBUG::2012-12-09 12:59:50,211::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2472::DEBUG::2012-12-09 12:59:50,211::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2472::DEBUG::2012-12-09 12:59:50,212::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2472::DEBUG::2012-12-09 12:59:50,212::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2472::DEBUG::2012-12-09 12:59:50,212::task::978::TaskManager.Task::(_decref) Task=`cbe0e081-4192-47fc-b798-17f8b8482610`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 12:59:51,393::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:51,434::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169554 s, 60.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 12:59:52,359::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 12:59:52,370::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000481624 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:53,442::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:53,482::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159263 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:55,488::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:55,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01762 s, 58.1 MB/s\n'; <rc> = 0
>Thread-2476::DEBUG::2012-12-09 12:59:56,925::task::588::TaskManager.Task::(_updateState) Task=`f7f9da03-76be-40bd-900b-10fdbcf93c5a`::moving from state init -> state preparing
>Thread-2476::INFO::2012-12-09 12:59:56,926::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2476::INFO::2012-12-09 12:59:56,926::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0110831260681', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2476::DEBUG::2012-12-09 12:59:56,926::task::1172::TaskManager.Task::(prepare) Task=`f7f9da03-76be-40bd-900b-10fdbcf93c5a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0110831260681', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2476::DEBUG::2012-12-09 12:59:56,926::task::588::TaskManager.Task::(_updateState) Task=`f7f9da03-76be-40bd-900b-10fdbcf93c5a`::moving from state preparing -> state finished
>Thread-2476::DEBUG::2012-12-09 12:59:56,926::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2476::DEBUG::2012-12-09 12:59:56,927::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2476::DEBUG::2012-12-09 12:59:56,927::task::978::TaskManager.Task::(_decref) Task=`f7f9da03-76be-40bd-900b-10fdbcf93c5a`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 12:59:57,076::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 12:59:57,077::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 12:59:57,174::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 12:59:57,175::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 12:59:57,175::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 12:59:57,176::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 12:59:57,538::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:57,578::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167344 s, 61.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 12:59:59,585::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 12:59:59,624::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166001 s, 61.7 MB/s\n'; <rc> = 0
>Thread-2479::DEBUG::2012-12-09 13:00:01,587::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2479::DEBUG::2012-12-09 13:00:01,588::task::588::TaskManager.Task::(_updateState) Task=`ef12e197-a4d5-444f-93a9-61c8607fa860`::moving from state init -> state preparing
>Thread-2479::INFO::2012-12-09 13:00:01,588::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2479::INFO::2012-12-09 13:00:01,589::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2479::DEBUG::2012-12-09 13:00:01,589::task::1172::TaskManager.Task::(prepare) Task=`ef12e197-a4d5-444f-93a9-61c8607fa860`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2479::DEBUG::2012-12-09 13:00:01,589::task::588::TaskManager.Task::(_updateState) Task=`ef12e197-a4d5-444f-93a9-61c8607fa860`::moving from state preparing -> state finished
>Thread-2479::DEBUG::2012-12-09 13:00:01,589::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2479::DEBUG::2012-12-09 13:00:01,589::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2479::DEBUG::2012-12-09 13:00:01,590::task::978::TaskManager.Task::(_decref) Task=`ef12e197-a4d5-444f-93a9-61c8607fa860`::ref 0 aborting False
>Thread-2480::DEBUG::2012-12-09 13:00:01,608::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2480::DEBUG::2012-12-09 13:00:01,608::task::588::TaskManager.Task::(_updateState) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::moving from state init -> state preparing
>Thread-2480::INFO::2012-12-09 13:00:01,608::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2480::DEBUG::2012-12-09 13:00:01,609::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`856348c9-9d4f-4f8c-8b81-fa48c5454775`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2480::DEBUG::2012-12-09 13:00:01,609::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2480::DEBUG::2012-12-09 13:00:01,609::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2480::DEBUG::2012-12-09 13:00:01,609::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`856348c9-9d4f-4f8c-8b81-fa48c5454775`::Granted request
>Thread-2480::DEBUG::2012-12-09 13:00:01,610::task::817::TaskManager.Task::(resourceAcquired) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2480::DEBUG::2012-12-09 13:00:01,610::task::978::TaskManager.Task::(_decref) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::ref 1 aborting False
>Thread-2480::DEBUG::2012-12-09 13:00:01,611::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2480::DEBUG::2012-12-09 13:00:01,611::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:01,631::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:01,671::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016356 s, 62.6 MB/s\n'; <rc> = 0
>Thread-2480::DEBUG::2012-12-09 13:00:01,713::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2480::WARNING::2012-12-09 13:00:01,714::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2480::DEBUG::2012-12-09 13:00:01,714::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2480::WARNING::2012-12-09 13:00:01,715::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2480::INFO::2012-12-09 13:00:01,715::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2480::DEBUG::2012-12-09 13:00:01,715::task::1172::TaskManager.Task::(prepare) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2480::DEBUG::2012-12-09 13:00:01,715::task::588::TaskManager.Task::(_updateState) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::moving from state preparing -> state finished
>Thread-2480::DEBUG::2012-12-09 13:00:01,716::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2480::DEBUG::2012-12-09 13:00:01,716::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2480::DEBUG::2012-12-09 13:00:01,716::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2480::DEBUG::2012-12-09 13:00:01,716::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2480::DEBUG::2012-12-09 13:00:01,716::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2480::DEBUG::2012-12-09 13:00:01,717::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2480::DEBUG::2012-12-09 13:00:01,717::task::978::TaskManager.Task::(_decref) Task=`08057fb7-c295-4019-a9c2-3ee44dd7b205`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:00:02,373::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:02,381::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000554473 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:03,677::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:03,717::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167037 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:05,724::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:05,763::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166148 s, 61.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:07,176::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:07,177::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:07,278::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:07,279::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:07,280::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:07,280::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:07,770::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:07,810::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158743 s, 64.5 MB/s\n'; <rc> = 0
>Thread-2484::DEBUG::2012-12-09 13:00:07,897::task::588::TaskManager.Task::(_updateState) Task=`83f68b73-eeb0-4614-a52a-2a8f761f96d4`::moving from state init -> state preparing
>Thread-2484::INFO::2012-12-09 13:00:07,897::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2484::INFO::2012-12-09 13:00:07,898::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00827097892761', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.6', 'code': 358, 'valid': False}}
>Thread-2484::DEBUG::2012-12-09 13:00:07,898::task::1172::TaskManager.Task::(prepare) Task=`83f68b73-eeb0-4614-a52a-2a8f761f96d4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00827097892761', 'lastCheck': '5.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.6', 'code': 358, 'valid': False}}
>Thread-2484::DEBUG::2012-12-09 13:00:07,898::task::588::TaskManager.Task::(_updateState) Task=`83f68b73-eeb0-4614-a52a-2a8f761f96d4`::moving from state preparing -> state finished
>Thread-2484::DEBUG::2012-12-09 13:00:07,898::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2484::DEBUG::2012-12-09 13:00:07,899::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2484::DEBUG::2012-12-09 13:00:07,899::task::978::TaskManager.Task::(_decref) Task=`83f68b73-eeb0-4614-a52a-2a8f761f96d4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:09,817::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:09,856::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158621 s, 64.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:11,863::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:11,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187821 s, 54.5 MB/s\n'; <rc> = 0
>Thread-2487::DEBUG::2012-12-09 13:00:12,077::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2487::DEBUG::2012-12-09 13:00:12,078::task::588::TaskManager.Task::(_updateState) Task=`59c169db-069d-4d96-9ab9-283e4854ea5b`::moving from state init -> state preparing
>Thread-2487::INFO::2012-12-09 13:00:12,079::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2487::INFO::2012-12-09 13:00:12,080::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2487::DEBUG::2012-12-09 13:00:12,080::task::1172::TaskManager.Task::(prepare) Task=`59c169db-069d-4d96-9ab9-283e4854ea5b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2487::DEBUG::2012-12-09 13:00:12,080::task::588::TaskManager.Task::(_updateState) Task=`59c169db-069d-4d96-9ab9-283e4854ea5b`::moving from state preparing -> state finished
>Thread-2487::DEBUG::2012-12-09 13:00:12,081::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2487::DEBUG::2012-12-09 13:00:12,081::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2487::DEBUG::2012-12-09 13:00:12,082::task::978::TaskManager.Task::(_decref) Task=`59c169db-069d-4d96-9ab9-283e4854ea5b`::ref 0 aborting False
>Thread-2488::DEBUG::2012-12-09 13:00:12,097::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2488::DEBUG::2012-12-09 13:00:12,097::task::588::TaskManager.Task::(_updateState) Task=`c1daa317-3d57-4719-90da-713df4d44879`::moving from state init -> state preparing
>Thread-2488::INFO::2012-12-09 13:00:12,097::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2488::DEBUG::2012-12-09 13:00:12,098::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`37629d37-e920-48fa-858a-5132b93547a5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2488::DEBUG::2012-12-09 13:00:12,098::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2488::DEBUG::2012-12-09 13:00:12,098::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2488::DEBUG::2012-12-09 13:00:12,098::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`37629d37-e920-48fa-858a-5132b93547a5`::Granted request
>Thread-2488::DEBUG::2012-12-09 13:00:12,099::task::817::TaskManager.Task::(resourceAcquired) Task=`c1daa317-3d57-4719-90da-713df4d44879`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2488::DEBUG::2012-12-09 13:00:12,099::task::978::TaskManager.Task::(_decref) Task=`c1daa317-3d57-4719-90da-713df4d44879`::ref 1 aborting False
>Thread-2488::DEBUG::2012-12-09 13:00:12,100::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2488::DEBUG::2012-12-09 13:00:12,100::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2488::DEBUG::2012-12-09 13:00:12,203::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2488::WARNING::2012-12-09 13:00:12,204::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2488::DEBUG::2012-12-09 13:00:12,204::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2488::WARNING::2012-12-09 13:00:12,205::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2488::INFO::2012-12-09 13:00:12,205::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2488::DEBUG::2012-12-09 13:00:12,206::task::1172::TaskManager.Task::(prepare) Task=`c1daa317-3d57-4719-90da-713df4d44879`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2488::DEBUG::2012-12-09 13:00:12,206::task::588::TaskManager.Task::(_updateState) Task=`c1daa317-3d57-4719-90da-713df4d44879`::moving from state preparing -> state finished
>Thread-2488::DEBUG::2012-12-09 13:00:12,206::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2488::DEBUG::2012-12-09 13:00:12,206::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2488::DEBUG::2012-12-09 13:00:12,207::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2488::DEBUG::2012-12-09 13:00:12,207::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2488::DEBUG::2012-12-09 13:00:12,207::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2488::DEBUG::2012-12-09 13:00:12,207::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2488::DEBUG::2012-12-09 13:00:12,207::task::978::TaskManager.Task::(_decref) Task=`c1daa317-3d57-4719-90da-713df4d44879`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:00:12,384::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:12,397::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000496684 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:13,910::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:13,958::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197717 s, 51.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:15,965::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:16,008::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176226 s, 58.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:17,281::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:17,282::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:17,400::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:17,401::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:17,401::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:17,402::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:18,015::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:18,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266822 s, 38.4 MB/s\n'; <rc> = 0
>Thread-2492::DEBUG::2012-12-09 13:00:18,382::task::588::TaskManager.Task::(_updateState) Task=`a6f3f5f9-746d-48a4-aa3d-1dafd725a218`::moving from state init -> state preparing
>Thread-2492::INFO::2012-12-09 13:00:18,382::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2492::INFO::2012-12-09 13:00:18,383::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141401290894', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-2492::DEBUG::2012-12-09 13:00:18,383::task::1172::TaskManager.Task::(prepare) Task=`a6f3f5f9-746d-48a4-aa3d-1dafd725a218`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141401290894', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-2492::DEBUG::2012-12-09 13:00:18,383::task::588::TaskManager.Task::(_updateState) Task=`a6f3f5f9-746d-48a4-aa3d-1dafd725a218`::moving from state preparing -> state finished
>Thread-2492::DEBUG::2012-12-09 13:00:18,384::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2492::DEBUG::2012-12-09 13:00:18,384::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2492::DEBUG::2012-12-09 13:00:18,384::task::978::TaskManager.Task::(_decref) Task=`a6f3f5f9-746d-48a4-aa3d-1dafd725a218`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:20,074::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:20,114::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161912 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:22,121::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:22,179::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254282 s, 40.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:00:22,401::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:22,413::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000468679 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:24,187::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:24,229::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179144 s, 57.2 MB/s\n'; <rc> = 0
>Thread-2496::DEBUG::2012-12-09 13:00:24,463::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2496::DEBUG::2012-12-09 13:00:24,463::task::588::TaskManager.Task::(_updateState) Task=`591359b3-3706-47e5-865b-6f3ef7c38940`::moving from state init -> state preparing
>Thread-2496::INFO::2012-12-09 13:00:24,464::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2496::INFO::2012-12-09 13:00:24,464::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2496::DEBUG::2012-12-09 13:00:24,464::task::1172::TaskManager.Task::(prepare) Task=`591359b3-3706-47e5-865b-6f3ef7c38940`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2496::DEBUG::2012-12-09 13:00:24,464::task::588::TaskManager.Task::(_updateState) Task=`591359b3-3706-47e5-865b-6f3ef7c38940`::moving from state preparing -> state finished
>Thread-2496::DEBUG::2012-12-09 13:00:24,465::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2496::DEBUG::2012-12-09 13:00:24,465::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2496::DEBUG::2012-12-09 13:00:24,465::task::978::TaskManager.Task::(_decref) Task=`591359b3-3706-47e5-865b-6f3ef7c38940`::ref 0 aborting False
>Thread-2497::DEBUG::2012-12-09 13:00:24,481::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2497::DEBUG::2012-12-09 13:00:24,482::task::588::TaskManager.Task::(_updateState) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::moving from state init -> state preparing
>Thread-2497::INFO::2012-12-09 13:00:24,482::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2497::DEBUG::2012-12-09 13:00:24,483::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`143d9148-a64c-4060-a92e-feaa2cf58be8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2497::DEBUG::2012-12-09 13:00:24,483::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2497::DEBUG::2012-12-09 13:00:24,483::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2497::DEBUG::2012-12-09 13:00:24,484::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`143d9148-a64c-4060-a92e-feaa2cf58be8`::Granted request
>Thread-2497::DEBUG::2012-12-09 13:00:24,484::task::817::TaskManager.Task::(resourceAcquired) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2497::DEBUG::2012-12-09 13:00:24,485::task::978::TaskManager.Task::(_decref) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::ref 1 aborting False
>Thread-2497::DEBUG::2012-12-09 13:00:24,485::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2497::DEBUG::2012-12-09 13:00:24,486::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2497::DEBUG::2012-12-09 13:00:24,594::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2497::WARNING::2012-12-09 13:00:24,595::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2497::DEBUG::2012-12-09 13:00:24,596::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2497::WARNING::2012-12-09 13:00:24,596::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2497::INFO::2012-12-09 13:00:24,597::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2497::DEBUG::2012-12-09 13:00:24,597::task::1172::TaskManager.Task::(prepare) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2497::DEBUG::2012-12-09 13:00:24,597::task::588::TaskManager.Task::(_updateState) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::moving from state preparing -> state finished
>Thread-2497::DEBUG::2012-12-09 13:00:24,597::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2497::DEBUG::2012-12-09 13:00:24,597::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2497::DEBUG::2012-12-09 13:00:24,598::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2497::DEBUG::2012-12-09 13:00:24,598::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2497::DEBUG::2012-12-09 13:00:24,598::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2497::DEBUG::2012-12-09 13:00:24,598::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2497::DEBUG::2012-12-09 13:00:24,599::task::978::TaskManager.Task::(_decref) Task=`4023d431-9cbb-4b44-93cf-e90075d1b150`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:26,236::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:26,276::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016106 s, 63.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:27,403::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:27,403::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:27,501::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:27,502::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:27,502::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:27,503::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:28,282::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:28,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159364 s, 64.3 MB/s\n'; <rc> = 0
>Thread-2500::DEBUG::2012-12-09 13:00:28,832::task::588::TaskManager.Task::(_updateState) Task=`82943425-7e79-459d-ab29-21841adfb4d6`::moving from state init -> state preparing
>Thread-2500::INFO::2012-12-09 13:00:28,832::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2500::INFO::2012-12-09 13:00:28,833::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120520591736', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-2500::DEBUG::2012-12-09 13:00:28,833::task::1172::TaskManager.Task::(prepare) Task=`82943425-7e79-459d-ab29-21841adfb4d6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120520591736', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False}}
>Thread-2500::DEBUG::2012-12-09 13:00:28,833::task::588::TaskManager.Task::(_updateState) Task=`82943425-7e79-459d-ab29-21841adfb4d6`::moving from state preparing -> state finished
>Thread-2500::DEBUG::2012-12-09 13:00:28,833::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2500::DEBUG::2012-12-09 13:00:28,833::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2500::DEBUG::2012-12-09 13:00:28,834::task::978::TaskManager.Task::(_decref) Task=`82943425-7e79-459d-ab29-21841adfb4d6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:30,327::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:30,367::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163839 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:32,374::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:32,416::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:32,427::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000480536 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:32,446::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257839 s, 39.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:34,453::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:34,496::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169542 s, 60.4 MB/s\n'; <rc> = 0
>Thread-2504::DEBUG::2012-12-09 13:00:34,667::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2504::DEBUG::2012-12-09 13:00:34,667::task::588::TaskManager.Task::(_updateState) Task=`1e682b65-8f71-40d5-936f-9984bb2c51f0`::moving from state init -> state preparing
>Thread-2504::INFO::2012-12-09 13:00:34,668::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2504::INFO::2012-12-09 13:00:34,668::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2504::DEBUG::2012-12-09 13:00:34,669::task::1172::TaskManager.Task::(prepare) Task=`1e682b65-8f71-40d5-936f-9984bb2c51f0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2504::DEBUG::2012-12-09 13:00:34,669::task::588::TaskManager.Task::(_updateState) Task=`1e682b65-8f71-40d5-936f-9984bb2c51f0`::moving from state preparing -> state finished
>Thread-2504::DEBUG::2012-12-09 13:00:34,669::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2504::DEBUG::2012-12-09 13:00:34,669::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2504::DEBUG::2012-12-09 13:00:34,669::task::978::TaskManager.Task::(_decref) Task=`1e682b65-8f71-40d5-936f-9984bb2c51f0`::ref 0 aborting False
>Thread-2505::DEBUG::2012-12-09 13:00:34,690::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2505::DEBUG::2012-12-09 13:00:34,691::task::588::TaskManager.Task::(_updateState) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::moving from state init -> state preparing
>Thread-2505::INFO::2012-12-09 13:00:34,691::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2505::DEBUG::2012-12-09 13:00:34,691::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a2cf3c7-1811-4cf5-a151-8b079cefb71d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2505::DEBUG::2012-12-09 13:00:34,692::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2505::DEBUG::2012-12-09 13:00:34,692::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2505::DEBUG::2012-12-09 13:00:34,692::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a2cf3c7-1811-4cf5-a151-8b079cefb71d`::Granted request
>Thread-2505::DEBUG::2012-12-09 13:00:34,693::task::817::TaskManager.Task::(resourceAcquired) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2505::DEBUG::2012-12-09 13:00:34,693::task::978::TaskManager.Task::(_decref) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::ref 1 aborting False
>Thread-2505::DEBUG::2012-12-09 13:00:34,694::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2505::DEBUG::2012-12-09 13:00:34,694::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2505::DEBUG::2012-12-09 13:00:34,799::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2505::WARNING::2012-12-09 13:00:34,802::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2505::DEBUG::2012-12-09 13:00:34,802::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2505::WARNING::2012-12-09 13:00:34,803::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2505::INFO::2012-12-09 13:00:34,804::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2505::DEBUG::2012-12-09 13:00:34,805::task::1172::TaskManager.Task::(prepare) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2505::DEBUG::2012-12-09 13:00:34,805::task::588::TaskManager.Task::(_updateState) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::moving from state preparing -> state finished
>Thread-2505::DEBUG::2012-12-09 13:00:34,806::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2505::DEBUG::2012-12-09 13:00:34,806::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2505::DEBUG::2012-12-09 13:00:34,807::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2505::DEBUG::2012-12-09 13:00:34,807::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2505::DEBUG::2012-12-09 13:00:34,808::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2505::DEBUG::2012-12-09 13:00:34,808::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2505::DEBUG::2012-12-09 13:00:34,809::task::978::TaskManager.Task::(_decref) Task=`dcd6379d-455f-46c2-9da1-734668b3c847`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:36,503::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:36,553::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216293 s, 47.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:37,504::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:37,504::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:37,606::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:37,607::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:37,607::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:37,608::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:38,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:38,603::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196463 s, 52.1 MB/s\n'; <rc> = 0
>Thread-2508::DEBUG::2012-12-09 13:00:39,117::task::588::TaskManager.Task::(_updateState) Task=`9301e1a5-1edd-41cd-bc5f-ea1881170b92`::moving from state init -> state preparing
>Thread-2508::INFO::2012-12-09 13:00:39,118::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2508::INFO::2012-12-09 13:00:39,118::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124368667603', 'lastCheck': '6.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-2508::DEBUG::2012-12-09 13:00:39,118::task::1172::TaskManager.Task::(prepare) Task=`9301e1a5-1edd-41cd-bc5f-ea1881170b92`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124368667603', 'lastCheck': '6.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-2508::DEBUG::2012-12-09 13:00:39,119::task::588::TaskManager.Task::(_updateState) Task=`9301e1a5-1edd-41cd-bc5f-ea1881170b92`::moving from state preparing -> state finished
>Thread-2508::DEBUG::2012-12-09 13:00:39,119::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2508::DEBUG::2012-12-09 13:00:39,119::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2508::DEBUG::2012-12-09 13:00:39,119::task::978::TaskManager.Task::(_decref) Task=`9301e1a5-1edd-41cd-bc5f-ea1881170b92`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:40,611::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:40,654::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183699 s, 55.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:00:42,433::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:42,442::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000430974 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:42,661::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:42,700::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159375 s, 64.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:44,707::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:44,746::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01577 s, 64.9 MB/s\n'; <rc> = 0
>Thread-2513::DEBUG::2012-12-09 13:00:45,698::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2513::DEBUG::2012-12-09 13:00:45,699::task::588::TaskManager.Task::(_updateState) Task=`4698c440-a182-425c-bb4b-566751602ce3`::moving from state init -> state preparing
>Thread-2513::INFO::2012-12-09 13:00:45,699::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2513::INFO::2012-12-09 13:00:45,700::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2513::DEBUG::2012-12-09 13:00:45,700::task::1172::TaskManager.Task::(prepare) Task=`4698c440-a182-425c-bb4b-566751602ce3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2513::DEBUG::2012-12-09 13:00:45,700::task::588::TaskManager.Task::(_updateState) Task=`4698c440-a182-425c-bb4b-566751602ce3`::moving from state preparing -> state finished
>Thread-2513::DEBUG::2012-12-09 13:00:45,700::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2513::DEBUG::2012-12-09 13:00:45,700::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2513::DEBUG::2012-12-09 13:00:45,701::task::978::TaskManager.Task::(_decref) Task=`4698c440-a182-425c-bb4b-566751602ce3`::ref 0 aborting False
>Thread-2514::DEBUG::2012-12-09 13:00:45,713::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2514::DEBUG::2012-12-09 13:00:45,713::task::588::TaskManager.Task::(_updateState) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::moving from state init -> state preparing
>Thread-2514::INFO::2012-12-09 13:00:45,713::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2514::DEBUG::2012-12-09 13:00:45,714::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6d74d768-08a7-42c3-bee1-c2d777042713`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2514::DEBUG::2012-12-09 13:00:45,714::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2514::DEBUG::2012-12-09 13:00:45,714::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2514::DEBUG::2012-12-09 13:00:45,714::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6d74d768-08a7-42c3-bee1-c2d777042713`::Granted request
>Thread-2514::DEBUG::2012-12-09 13:00:45,715::task::817::TaskManager.Task::(resourceAcquired) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2514::DEBUG::2012-12-09 13:00:45,715::task::978::TaskManager.Task::(_decref) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::ref 1 aborting False
>Thread-2514::DEBUG::2012-12-09 13:00:45,716::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2514::DEBUG::2012-12-09 13:00:45,717::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2514::DEBUG::2012-12-09 13:00:45,810::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2514::WARNING::2012-12-09 13:00:45,812::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2514::DEBUG::2012-12-09 13:00:45,812::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2514::WARNING::2012-12-09 13:00:45,812::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2514::INFO::2012-12-09 13:00:45,813::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2514::DEBUG::2012-12-09 13:00:45,813::task::1172::TaskManager.Task::(prepare) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2514::DEBUG::2012-12-09 13:00:45,813::task::588::TaskManager.Task::(_updateState) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::moving from state preparing -> state finished
>Thread-2514::DEBUG::2012-12-09 13:00:45,813::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2514::DEBUG::2012-12-09 13:00:45,813::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2514::DEBUG::2012-12-09 13:00:45,814::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2514::DEBUG::2012-12-09 13:00:45,814::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2514::DEBUG::2012-12-09 13:00:45,814::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2514::DEBUG::2012-12-09 13:00:45,814::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2514::DEBUG::2012-12-09 13:00:45,814::task::978::TaskManager.Task::(_decref) Task=`d6d64693-0f45-4e47-bdff-df1efbbd718a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:46,754::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:46,794::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174314 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:47,609::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:47,609::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:47,708::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:47,709::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:47,709::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:47,709::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:48,801::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:48,839::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161807 s, 63.3 MB/s\n'; <rc> = 0
>Thread-2516::DEBUG::2012-12-09 13:00:49,419::task::588::TaskManager.Task::(_updateState) Task=`11b3aab7-4e87-42f6-b307-12384cf2d0e6`::moving from state init -> state preparing
>Thread-2516::INFO::2012-12-09 13:00:49,420::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2516::INFO::2012-12-09 13:00:49,420::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00916409492493', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2516::DEBUG::2012-12-09 13:00:49,420::task::1172::TaskManager.Task::(prepare) Task=`11b3aab7-4e87-42f6-b307-12384cf2d0e6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00916409492493', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2516::DEBUG::2012-12-09 13:00:49,421::task::588::TaskManager.Task::(_updateState) Task=`11b3aab7-4e87-42f6-b307-12384cf2d0e6`::moving from state preparing -> state finished
>Thread-2516::DEBUG::2012-12-09 13:00:49,421::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2516::DEBUG::2012-12-09 13:00:49,421::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2516::DEBUG::2012-12-09 13:00:49,421::task::978::TaskManager.Task::(_decref) Task=`11b3aab7-4e87-42f6-b307-12384cf2d0e6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:50,846::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:50,886::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161288 s, 63.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:00:52,445::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:00:52,458::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000845179 s, 4.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:52,893::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:52,932::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01603 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:00:54,939::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:54,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158985 s, 64.4 MB/s\n'; <rc> = 0
>Thread-2521::DEBUG::2012-12-09 13:00:56,107::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2521::DEBUG::2012-12-09 13:00:56,107::task::588::TaskManager.Task::(_updateState) Task=`263ecca0-4cee-4ee0-91ed-a7df1f8fc54a`::moving from state init -> state preparing
>Thread-2521::INFO::2012-12-09 13:00:56,108::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2521::INFO::2012-12-09 13:00:56,108::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2521::DEBUG::2012-12-09 13:00:56,108::task::1172::TaskManager.Task::(prepare) Task=`263ecca0-4cee-4ee0-91ed-a7df1f8fc54a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2521::DEBUG::2012-12-09 13:00:56,109::task::588::TaskManager.Task::(_updateState) Task=`263ecca0-4cee-4ee0-91ed-a7df1f8fc54a`::moving from state preparing -> state finished
>Thread-2521::DEBUG::2012-12-09 13:00:56,109::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2521::DEBUG::2012-12-09 13:00:56,109::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2521::DEBUG::2012-12-09 13:00:56,110::task::978::TaskManager.Task::(_decref) Task=`263ecca0-4cee-4ee0-91ed-a7df1f8fc54a`::ref 0 aborting False
>Thread-2522::DEBUG::2012-12-09 13:00:56,123::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2522::DEBUG::2012-12-09 13:00:56,123::task::588::TaskManager.Task::(_updateState) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::moving from state init -> state preparing
>Thread-2522::INFO::2012-12-09 13:00:56,124::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2522::DEBUG::2012-12-09 13:00:56,124::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`10f07ce7-518d-4816-8534-0759ba9e2c3e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2522::DEBUG::2012-12-09 13:00:56,125::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2522::DEBUG::2012-12-09 13:00:56,125::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2522::DEBUG::2012-12-09 13:00:56,125::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`10f07ce7-518d-4816-8534-0759ba9e2c3e`::Granted request
>Thread-2522::DEBUG::2012-12-09 13:00:56,126::task::817::TaskManager.Task::(resourceAcquired) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2522::DEBUG::2012-12-09 13:00:56,126::task::978::TaskManager.Task::(_decref) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::ref 1 aborting False
>Thread-2522::DEBUG::2012-12-09 13:00:56,127::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2522::DEBUG::2012-12-09 13:00:56,127::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2522::DEBUG::2012-12-09 13:00:56,227::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2522::WARNING::2012-12-09 13:00:56,229::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2522::DEBUG::2012-12-09 13:00:56,229::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2522::WARNING::2012-12-09 13:00:56,230::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2522::INFO::2012-12-09 13:00:56,230::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2522::DEBUG::2012-12-09 13:00:56,230::task::1172::TaskManager.Task::(prepare) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2522::DEBUG::2012-12-09 13:00:56,231::task::588::TaskManager.Task::(_updateState) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::moving from state preparing -> state finished
>Thread-2522::DEBUG::2012-12-09 13:00:56,231::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2522::DEBUG::2012-12-09 13:00:56,231::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2522::DEBUG::2012-12-09 13:00:56,232::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2522::DEBUG::2012-12-09 13:00:56,232::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2522::DEBUG::2012-12-09 13:00:56,232::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2522::DEBUG::2012-12-09 13:00:56,232::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2522::DEBUG::2012-12-09 13:00:56,232::task::978::TaskManager.Task::(_decref) Task=`c54658b9-0e89-4fb4-902e-ecc035ffa666`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:00:56,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:57,024::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162268 s, 63.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:00:57,710::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:00:57,711::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:00:57,809::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:00:57,810::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:00:57,810::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:00:57,810::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:00:59,031::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:00:59,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172736 s, 59.3 MB/s\n'; <rc> = 0
>Thread-2524::DEBUG::2012-12-09 13:01:00,091::task::588::TaskManager.Task::(_updateState) Task=`00c4b5af-b25b-45d5-90f1-f8ce898275c8`::moving from state init -> state preparing
>Thread-2524::INFO::2012-12-09 13:01:00,091::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2524::INFO::2012-12-09 13:01:00,091::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141520500183', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-2524::DEBUG::2012-12-09 13:01:00,092::task::1172::TaskManager.Task::(prepare) Task=`00c4b5af-b25b-45d5-90f1-f8ce898275c8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0141520500183', 'lastCheck': '7.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.3', 'code': 358, 'valid': False}}
>Thread-2524::DEBUG::2012-12-09 13:01:00,092::task::588::TaskManager.Task::(_updateState) Task=`00c4b5af-b25b-45d5-90f1-f8ce898275c8`::moving from state preparing -> state finished
>Thread-2524::DEBUG::2012-12-09 13:01:00,092::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2524::DEBUG::2012-12-09 13:01:00,092::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2524::DEBUG::2012-12-09 13:01:00,092::task::978::TaskManager.Task::(_decref) Task=`00c4b5af-b25b-45d5-90f1-f8ce898275c8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:01,085::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:01,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175554 s, 58.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:02,464::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:02,472::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000345214 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:03,145::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:03,187::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182469 s, 56.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:05,193::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:05,232::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162358 s, 63.1 MB/s\n'; <rc> = 0
>Thread-2528::DEBUG::2012-12-09 13:01:06,448::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2528::DEBUG::2012-12-09 13:01:06,448::task::588::TaskManager.Task::(_updateState) Task=`22835c04-ba53-4e85-8398-b69f06bc8f1c`::moving from state init -> state preparing
>Thread-2528::INFO::2012-12-09 13:01:06,448::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2528::INFO::2012-12-09 13:01:06,449::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2528::DEBUG::2012-12-09 13:01:06,449::task::1172::TaskManager.Task::(prepare) Task=`22835c04-ba53-4e85-8398-b69f06bc8f1c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2528::DEBUG::2012-12-09 13:01:06,449::task::588::TaskManager.Task::(_updateState) Task=`22835c04-ba53-4e85-8398-b69f06bc8f1c`::moving from state preparing -> state finished
>Thread-2528::DEBUG::2012-12-09 13:01:06,449::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2528::DEBUG::2012-12-09 13:01:06,450::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2528::DEBUG::2012-12-09 13:01:06,450::task::978::TaskManager.Task::(_decref) Task=`22835c04-ba53-4e85-8398-b69f06bc8f1c`::ref 0 aborting False
>Thread-2529::DEBUG::2012-12-09 13:01:06,462::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2529::DEBUG::2012-12-09 13:01:06,463::task::588::TaskManager.Task::(_updateState) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::moving from state init -> state preparing
>Thread-2529::INFO::2012-12-09 13:01:06,463::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2529::DEBUG::2012-12-09 13:01:06,464::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`41df3097-3353-42d7-b805-c54b436c330e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2529::DEBUG::2012-12-09 13:01:06,464::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2529::DEBUG::2012-12-09 13:01:06,464::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2529::DEBUG::2012-12-09 13:01:06,464::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`41df3097-3353-42d7-b805-c54b436c330e`::Granted request
>Thread-2529::DEBUG::2012-12-09 13:01:06,465::task::817::TaskManager.Task::(resourceAcquired) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2529::DEBUG::2012-12-09 13:01:06,465::task::978::TaskManager.Task::(_decref) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::ref 1 aborting False
>Thread-2529::DEBUG::2012-12-09 13:01:06,466::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2529::DEBUG::2012-12-09 13:01:06,466::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2529::DEBUG::2012-12-09 13:01:06,570::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2529::WARNING::2012-12-09 13:01:06,572::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2529::DEBUG::2012-12-09 13:01:06,572::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2529::WARNING::2012-12-09 13:01:06,573::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2529::INFO::2012-12-09 13:01:06,573::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2529::DEBUG::2012-12-09 13:01:06,573::task::1172::TaskManager.Task::(prepare) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2529::DEBUG::2012-12-09 13:01:06,573::task::588::TaskManager.Task::(_updateState) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::moving from state preparing -> state finished
>Thread-2529::DEBUG::2012-12-09 13:01:06,574::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2529::DEBUG::2012-12-09 13:01:06,574::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2529::DEBUG::2012-12-09 13:01:06,574::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2529::DEBUG::2012-12-09 13:01:06,574::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2529::DEBUG::2012-12-09 13:01:06,575::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2529::DEBUG::2012-12-09 13:01:06,575::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2529::DEBUG::2012-12-09 13:01:06,575::task::978::TaskManager.Task::(_decref) Task=`cd4f6cec-b5ec-4c27-bf8f-8a8654684714`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:07,239::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:07,278::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162809 s, 62.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:07,811::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:07,812::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:01:07,909::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:07,910::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:07,910::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:07,911::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:01:09,285::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:09,330::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209591 s, 48.9 MB/s\n'; <rc> = 0
>Thread-2532::DEBUG::2012-12-09 13:01:10,943::task::588::TaskManager.Task::(_updateState) Task=`4cbd3ecf-1f07-4d89-a962-9fb3aa4b544e`::moving from state init -> state preparing
>Thread-2532::INFO::2012-12-09 13:01:10,944::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2532::INFO::2012-12-09 13:01:10,944::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00842308998108', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-2532::DEBUG::2012-12-09 13:01:10,944::task::1172::TaskManager.Task::(prepare) Task=`4cbd3ecf-1f07-4d89-a962-9fb3aa4b544e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00842308998108', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.0', 'code': 358, 'valid': False}}
>Thread-2532::DEBUG::2012-12-09 13:01:10,944::task::588::TaskManager.Task::(_updateState) Task=`4cbd3ecf-1f07-4d89-a962-9fb3aa4b544e`::moving from state preparing -> state finished
>Thread-2532::DEBUG::2012-12-09 13:01:10,945::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2532::DEBUG::2012-12-09 13:01:10,945::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2532::DEBUG::2012-12-09 13:01:10,945::task::978::TaskManager.Task::(_decref) Task=`4cbd3ecf-1f07-4d89-a962-9fb3aa4b544e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:11,337::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:11,378::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170362 s, 60.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:12,475::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:12,490::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000741488 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:13,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:13,434::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222431 s, 46.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:15,441::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:15,483::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171553 s, 59.7 MB/s\n'; <rc> = 0
>Thread-2536::DEBUG::2012-12-09 13:01:17,445::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2536::DEBUG::2012-12-09 13:01:17,445::task::588::TaskManager.Task::(_updateState) Task=`f6683bbf-8e82-4750-92da-18ea82ce24cc`::moving from state init -> state preparing
>Thread-2536::INFO::2012-12-09 13:01:17,446::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2536::INFO::2012-12-09 13:01:17,446::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2536::DEBUG::2012-12-09 13:01:17,446::task::1172::TaskManager.Task::(prepare) Task=`f6683bbf-8e82-4750-92da-18ea82ce24cc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2536::DEBUG::2012-12-09 13:01:17,446::task::588::TaskManager.Task::(_updateState) Task=`f6683bbf-8e82-4750-92da-18ea82ce24cc`::moving from state preparing -> state finished
>Thread-2536::DEBUG::2012-12-09 13:01:17,447::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2536::DEBUG::2012-12-09 13:01:17,447::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2536::DEBUG::2012-12-09 13:01:17,447::task::978::TaskManager.Task::(_decref) Task=`f6683bbf-8e82-4750-92da-18ea82ce24cc`::ref 0 aborting False
>Thread-2538::DEBUG::2012-12-09 13:01:17,469::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2538::DEBUG::2012-12-09 13:01:17,469::task::588::TaskManager.Task::(_updateState) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::moving from state init -> state preparing
>Thread-2538::INFO::2012-12-09 13:01:17,469::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2538::DEBUG::2012-12-09 13:01:17,470::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`83d86f0c-f809-418c-98d7-b7a78c702a13`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2538::DEBUG::2012-12-09 13:01:17,470::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2538::DEBUG::2012-12-09 13:01:17,470::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2538::DEBUG::2012-12-09 13:01:17,470::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`83d86f0c-f809-418c-98d7-b7a78c702a13`::Granted request
>Thread-2538::DEBUG::2012-12-09 13:01:17,471::task::817::TaskManager.Task::(resourceAcquired) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2538::DEBUG::2012-12-09 13:01:17,471::task::978::TaskManager.Task::(_decref) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::ref 1 aborting False
>Thread-2538::DEBUG::2012-12-09 13:01:17,472::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2538::DEBUG::2012-12-09 13:01:17,472::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:17,490::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:17,529::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162957 s, 62.8 MB/s\n'; <rc> = 0
>Thread-2538::DEBUG::2012-12-09 13:01:17,574::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2538::WARNING::2012-12-09 13:01:17,575::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2538::DEBUG::2012-12-09 13:01:17,575::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2538::WARNING::2012-12-09 13:01:17,575::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2538::INFO::2012-12-09 13:01:17,576::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2538::DEBUG::2012-12-09 13:01:17,576::task::1172::TaskManager.Task::(prepare) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2538::DEBUG::2012-12-09 13:01:17,576::task::588::TaskManager.Task::(_updateState) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::moving from state preparing -> state finished
>Thread-2538::DEBUG::2012-12-09 13:01:17,576::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2538::DEBUG::2012-12-09 13:01:17,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2538::DEBUG::2012-12-09 13:01:17,577::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2538::DEBUG::2012-12-09 13:01:17,577::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2538::DEBUG::2012-12-09 13:01:17,577::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2538::DEBUG::2012-12-09 13:01:17,578::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2538::DEBUG::2012-12-09 13:01:17,578::task::978::TaskManager.Task::(_decref) Task=`87cc2ff7-c600-4c0e-a311-eade72e0faa1`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:01:17,912::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:17,912::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:01:18,016::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:18,017::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:18,017::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:18,018::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:01:19,536::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:19,586::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024139 s, 42.4 MB/s\n'; <rc> = 0
>Thread-2540::DEBUG::2012-12-09 13:01:21,520::task::588::TaskManager.Task::(_updateState) Task=`700d5d86-f80b-41e0-b195-7b376a360f2e`::moving from state init -> state preparing
>Thread-2540::INFO::2012-12-09 13:01:21,521::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2540::INFO::2012-12-09 13:01:21,521::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0171740055084', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-2540::DEBUG::2012-12-09 13:01:21,521::task::1172::TaskManager.Task::(prepare) Task=`700d5d86-f80b-41e0-b195-7b376a360f2e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0171740055084', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.5', 'code': 358, 'valid': False}}
>Thread-2540::DEBUG::2012-12-09 13:01:21,521::task::588::TaskManager.Task::(_updateState) Task=`700d5d86-f80b-41e0-b195-7b376a360f2e`::moving from state preparing -> state finished
>Thread-2540::DEBUG::2012-12-09 13:01:21,522::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2540::DEBUG::2012-12-09 13:01:21,522::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2540::DEBUG::2012-12-09 13:01:21,522::task::978::TaskManager.Task::(_decref) Task=`700d5d86-f80b-41e0-b195-7b376a360f2e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:21,593::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:21,635::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166998 s, 61.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:22,495::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:22,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000547146 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:23,642::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:23,693::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247242 s, 41.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:25,700::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:25,740::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168844 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:27,747::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:27,789::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170258 s, 60.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:28,019::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:28,019::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2545::DEBUG::2012-12-09 13:01:28,090::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2545::DEBUG::2012-12-09 13:01:28,091::task::588::TaskManager.Task::(_updateState) Task=`492e6e7d-d83c-4628-8a56-a32de4bd7f0a`::moving from state init -> state preparing
>Thread-2545::INFO::2012-12-09 13:01:28,092::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2545::INFO::2012-12-09 13:01:28,092::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2545::DEBUG::2012-12-09 13:01:28,093::task::1172::TaskManager.Task::(prepare) Task=`492e6e7d-d83c-4628-8a56-a32de4bd7f0a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2545::DEBUG::2012-12-09 13:01:28,093::task::588::TaskManager.Task::(_updateState) Task=`492e6e7d-d83c-4628-8a56-a32de4bd7f0a`::moving from state preparing -> state finished
>Thread-2545::DEBUG::2012-12-09 13:01:28,093::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2545::DEBUG::2012-12-09 13:01:28,094::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2545::DEBUG::2012-12-09 13:01:28,094::task::978::TaskManager.Task::(_decref) Task=`492e6e7d-d83c-4628-8a56-a32de4bd7f0a`::ref 0 aborting False
>Thread-2546::DEBUG::2012-12-09 13:01:28,107::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2546::DEBUG::2012-12-09 13:01:28,108::task::588::TaskManager.Task::(_updateState) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::moving from state init -> state preparing
>Thread-2546::INFO::2012-12-09 13:01:28,108::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2546::DEBUG::2012-12-09 13:01:28,108::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`96710982-9609-4337-ba00-6dbd95823949`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2546::DEBUG::2012-12-09 13:01:28,109::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2546::DEBUG::2012-12-09 13:01:28,109::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2546::DEBUG::2012-12-09 13:01:28,109::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`96710982-9609-4337-ba00-6dbd95823949`::Granted request
>Thread-2546::DEBUG::2012-12-09 13:01:28,110::task::817::TaskManager.Task::(resourceAcquired) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2546::DEBUG::2012-12-09 13:01:28,110::task::978::TaskManager.Task::(_decref) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 13:01:28,128::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:28,129::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:28,130::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:28,130::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2546::DEBUG::2012-12-09 13:01:28,131::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2546::DEBUG::2012-12-09 13:01:28,132::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2546::DEBUG::2012-12-09 13:01:28,232::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2546::WARNING::2012-12-09 13:01:28,233::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2546::DEBUG::2012-12-09 13:01:28,233::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2546::WARNING::2012-12-09 13:01:28,234::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2546::INFO::2012-12-09 13:01:28,234::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2546::DEBUG::2012-12-09 13:01:28,234::task::1172::TaskManager.Task::(prepare) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2546::DEBUG::2012-12-09 13:01:28,235::task::588::TaskManager.Task::(_updateState) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::moving from state preparing -> state finished
>Thread-2546::DEBUG::2012-12-09 13:01:28,235::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2546::DEBUG::2012-12-09 13:01:28,235::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2546::DEBUG::2012-12-09 13:01:28,235::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2546::DEBUG::2012-12-09 13:01:28,236::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2546::DEBUG::2012-12-09 13:01:28,236::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2546::DEBUG::2012-12-09 13:01:28,236::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2546::DEBUG::2012-12-09 13:01:28,236::task::978::TaskManager.Task::(_decref) Task=`3da2b8e1-89bd-4b22-aeea-ade0994aaf51`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:29,796::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:29,842::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020639 s, 49.6 MB/s\n'; <rc> = 0
>Thread-2548::DEBUG::2012-12-09 13:01:31,758::task::588::TaskManager.Task::(_updateState) Task=`9ae625f9-192a-4a16-9b26-9915f58c2be3`::moving from state init -> state preparing
>Thread-2548::INFO::2012-12-09 13:01:31,758::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2548::INFO::2012-12-09 13:01:31,759::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129570960999', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-2548::DEBUG::2012-12-09 13:01:31,759::task::1172::TaskManager.Task::(prepare) Task=`9ae625f9-192a-4a16-9b26-9915f58c2be3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129570960999', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-2548::DEBUG::2012-12-09 13:01:31,759::task::588::TaskManager.Task::(_updateState) Task=`9ae625f9-192a-4a16-9b26-9915f58c2be3`::moving from state preparing -> state finished
>Thread-2548::DEBUG::2012-12-09 13:01:31,759::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2548::DEBUG::2012-12-09 13:01:31,759::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2548::DEBUG::2012-12-09 13:01:31,760::task::978::TaskManager.Task::(_decref) Task=`9ae625f9-192a-4a16-9b26-9915f58c2be3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:31,849::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:31,888::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163895 s, 62.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:32,510::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:32,522::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000447764 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:33,895::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:33,934::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161433 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:35,941::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:35,983::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176741 s, 57.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:37,989::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:38,029::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167371 s, 61.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:38,132::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:38,132::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:01:38,230::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:38,232::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:38,232::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:38,232::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2552::DEBUG::2012-12-09 13:01:38,581::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2552::DEBUG::2012-12-09 13:01:38,582::task::588::TaskManager.Task::(_updateState) Task=`7d09dceb-64b7-4a32-a38c-380c05d04563`::moving from state init -> state preparing
>Thread-2552::INFO::2012-12-09 13:01:38,582::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2552::INFO::2012-12-09 13:01:38,582::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2552::DEBUG::2012-12-09 13:01:38,582::task::1172::TaskManager.Task::(prepare) Task=`7d09dceb-64b7-4a32-a38c-380c05d04563`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2552::DEBUG::2012-12-09 13:01:38,583::task::588::TaskManager.Task::(_updateState) Task=`7d09dceb-64b7-4a32-a38c-380c05d04563`::moving from state preparing -> state finished
>Thread-2552::DEBUG::2012-12-09 13:01:38,583::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2552::DEBUG::2012-12-09 13:01:38,583::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2552::DEBUG::2012-12-09 13:01:38,583::task::978::TaskManager.Task::(_decref) Task=`7d09dceb-64b7-4a32-a38c-380c05d04563`::ref 0 aborting False
>Thread-2553::DEBUG::2012-12-09 13:01:38,599::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2553::DEBUG::2012-12-09 13:01:38,600::task::588::TaskManager.Task::(_updateState) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::moving from state init -> state preparing
>Thread-2553::INFO::2012-12-09 13:01:38,600::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2553::DEBUG::2012-12-09 13:01:38,600::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`52f2da96-422b-4f5f-ac53-91f009c18de8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2553::DEBUG::2012-12-09 13:01:38,600::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2553::DEBUG::2012-12-09 13:01:38,601::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2553::DEBUG::2012-12-09 13:01:38,601::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`52f2da96-422b-4f5f-ac53-91f009c18de8`::Granted request
>Thread-2553::DEBUG::2012-12-09 13:01:38,601::task::817::TaskManager.Task::(resourceAcquired) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2553::DEBUG::2012-12-09 13:01:38,601::task::978::TaskManager.Task::(_decref) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::ref 1 aborting False
>Thread-2553::DEBUG::2012-12-09 13:01:38,602::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2553::DEBUG::2012-12-09 13:01:38,602::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2553::DEBUG::2012-12-09 13:01:38,702::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2553::WARNING::2012-12-09 13:01:38,703::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2553::DEBUG::2012-12-09 13:01:38,703::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2553::WARNING::2012-12-09 13:01:38,704::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2553::INFO::2012-12-09 13:01:38,704::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2553::DEBUG::2012-12-09 13:01:38,704::task::1172::TaskManager.Task::(prepare) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2553::DEBUG::2012-12-09 13:01:38,705::task::588::TaskManager.Task::(_updateState) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::moving from state preparing -> state finished
>Thread-2553::DEBUG::2012-12-09 13:01:38,705::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2553::DEBUG::2012-12-09 13:01:38,705::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2553::DEBUG::2012-12-09 13:01:38,705::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2553::DEBUG::2012-12-09 13:01:38,706::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2553::DEBUG::2012-12-09 13:01:38,706::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2553::DEBUG::2012-12-09 13:01:38,706::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2553::DEBUG::2012-12-09 13:01:38,706::task::978::TaskManager.Task::(_decref) Task=`4e6b6a88-f8b8-438c-9947-81ef99b2c985`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:40,036::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:40,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163484 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:42,084::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:42,132::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230429 s, 44.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:42,525::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:42,535::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000411391 s, 10.0 MB/s\n'; <rc> = 0
>Thread-2556::DEBUG::2012-12-09 13:01:43,814::task::588::TaskManager.Task::(_updateState) Task=`66029c14-535c-4b62-bd36-46ba090b5ccd`::moving from state init -> state preparing
>Thread-2556::INFO::2012-12-09 13:01:43,815::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2556::INFO::2012-12-09 13:01:43,815::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0105810165405', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.6', 'code': 358, 'valid': False}}
>Thread-2556::DEBUG::2012-12-09 13:01:43,815::task::1172::TaskManager.Task::(prepare) Task=`66029c14-535c-4b62-bd36-46ba090b5ccd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0105810165405', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.6', 'code': 358, 'valid': False}}
>Thread-2556::DEBUG::2012-12-09 13:01:43,815::task::588::TaskManager.Task::(_updateState) Task=`66029c14-535c-4b62-bd36-46ba090b5ccd`::moving from state preparing -> state finished
>Thread-2556::DEBUG::2012-12-09 13:01:43,815::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2556::DEBUG::2012-12-09 13:01:43,816::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2556::DEBUG::2012-12-09 13:01:43,816::task::978::TaskManager.Task::(_decref) Task=`66029c14-535c-4b62-bd36-46ba090b5ccd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:44,139::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:44,180::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162146 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:46,187::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:46,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159443 s, 64.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:48,233::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:48,234::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:48,238::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:48,280::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168161 s, 60.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:48,333::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:48,334::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:48,335::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:48,335::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2560::DEBUG::2012-12-09 13:01:49,110::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2560::DEBUG::2012-12-09 13:01:49,111::task::588::TaskManager.Task::(_updateState) Task=`cfc72621-b982-40fe-a498-f7ada10437b9`::moving from state init -> state preparing
>Thread-2560::INFO::2012-12-09 13:01:49,111::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2560::INFO::2012-12-09 13:01:49,112::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2560::DEBUG::2012-12-09 13:01:49,112::task::1172::TaskManager.Task::(prepare) Task=`cfc72621-b982-40fe-a498-f7ada10437b9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2560::DEBUG::2012-12-09 13:01:49,112::task::588::TaskManager.Task::(_updateState) Task=`cfc72621-b982-40fe-a498-f7ada10437b9`::moving from state preparing -> state finished
>Thread-2560::DEBUG::2012-12-09 13:01:49,113::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2560::DEBUG::2012-12-09 13:01:49,113::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2560::DEBUG::2012-12-09 13:01:49,113::task::978::TaskManager.Task::(_decref) Task=`cfc72621-b982-40fe-a498-f7ada10437b9`::ref 0 aborting False
>Thread-2561::DEBUG::2012-12-09 13:01:49,127::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2561::DEBUG::2012-12-09 13:01:49,128::task::588::TaskManager.Task::(_updateState) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::moving from state init -> state preparing
>Thread-2561::INFO::2012-12-09 13:01:49,128::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2561::DEBUG::2012-12-09 13:01:49,128::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d66017b3-7e82-41fe-970d-f0eface22f8f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2561::DEBUG::2012-12-09 13:01:49,129::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2561::DEBUG::2012-12-09 13:01:49,129::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2561::DEBUG::2012-12-09 13:01:49,129::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d66017b3-7e82-41fe-970d-f0eface22f8f`::Granted request
>Thread-2561::DEBUG::2012-12-09 13:01:49,130::task::817::TaskManager.Task::(resourceAcquired) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2561::DEBUG::2012-12-09 13:01:49,130::task::978::TaskManager.Task::(_decref) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::ref 1 aborting False
>Thread-2561::DEBUG::2012-12-09 13:01:49,131::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2561::DEBUG::2012-12-09 13:01:49,132::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2561::DEBUG::2012-12-09 13:01:49,230::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2561::WARNING::2012-12-09 13:01:49,231::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2561::DEBUG::2012-12-09 13:01:49,231::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2561::WARNING::2012-12-09 13:01:49,232::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2561::INFO::2012-12-09 13:01:49,232::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2561::DEBUG::2012-12-09 13:01:49,232::task::1172::TaskManager.Task::(prepare) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2561::DEBUG::2012-12-09 13:01:49,233::task::588::TaskManager.Task::(_updateState) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::moving from state preparing -> state finished
>Thread-2561::DEBUG::2012-12-09 13:01:49,233::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2561::DEBUG::2012-12-09 13:01:49,233::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2561::DEBUG::2012-12-09 13:01:49,233::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2561::DEBUG::2012-12-09 13:01:49,233::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2561::DEBUG::2012-12-09 13:01:49,234::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2561::DEBUG::2012-12-09 13:01:49,234::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2561::DEBUG::2012-12-09 13:01:49,234::task::978::TaskManager.Task::(_decref) Task=`ef42f45b-90f2-4717-bc73-470109aeba6e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:50,287::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:50,327::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163149 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:52,334::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:52,374::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162569 s, 63.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:01:52,538::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:01:52,546::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000488351 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:01:54,381::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:54,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159626 s, 64.1 MB/s\n'; <rc> = 0
>Thread-2564::DEBUG::2012-12-09 13:01:54,688::task::588::TaskManager.Task::(_updateState) Task=`12e25946-5298-43d8-a068-6768cce62213`::moving from state init -> state preparing
>Thread-2564::INFO::2012-12-09 13:01:54,688::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2564::INFO::2012-12-09 13:01:54,688::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00844788551331', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-2564::DEBUG::2012-12-09 13:01:54,689::task::1172::TaskManager.Task::(prepare) Task=`12e25946-5298-43d8-a068-6768cce62213`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00844788551331', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-2564::DEBUG::2012-12-09 13:01:54,689::task::588::TaskManager.Task::(_updateState) Task=`12e25946-5298-43d8-a068-6768cce62213`::moving from state preparing -> state finished
>Thread-2564::DEBUG::2012-12-09 13:01:54,689::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2564::DEBUG::2012-12-09 13:01:54,689::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2564::DEBUG::2012-12-09 13:01:54,690::task::978::TaskManager.Task::(_decref) Task=`12e25946-5298-43d8-a068-6768cce62213`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:01:56,428::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:56,467::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154051 s, 66.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:01:58,336::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:01:58,337::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:01:58,435::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:01:58,436::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:01:58,436::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:01:58,437::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:01:58,474::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:01:58,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164462 s, 62.3 MB/s\n'; <rc> = 0
>Thread-2568::DEBUG::2012-12-09 13:01:59,559::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2568::DEBUG::2012-12-09 13:01:59,560::task::588::TaskManager.Task::(_updateState) Task=`95929af0-f05a-4ed5-ba7c-79e1b3dc1346`::moving from state init -> state preparing
>Thread-2568::INFO::2012-12-09 13:01:59,560::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2568::INFO::2012-12-09 13:01:59,560::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2568::DEBUG::2012-12-09 13:01:59,561::task::1172::TaskManager.Task::(prepare) Task=`95929af0-f05a-4ed5-ba7c-79e1b3dc1346`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2568::DEBUG::2012-12-09 13:01:59,561::task::588::TaskManager.Task::(_updateState) Task=`95929af0-f05a-4ed5-ba7c-79e1b3dc1346`::moving from state preparing -> state finished
>Thread-2568::DEBUG::2012-12-09 13:01:59,561::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2568::DEBUG::2012-12-09 13:01:59,561::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2568::DEBUG::2012-12-09 13:01:59,562::task::978::TaskManager.Task::(_decref) Task=`95929af0-f05a-4ed5-ba7c-79e1b3dc1346`::ref 0 aborting False
>Thread-2569::DEBUG::2012-12-09 13:01:59,574::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2569::DEBUG::2012-12-09 13:01:59,574::task::588::TaskManager.Task::(_updateState) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::moving from state init -> state preparing
>Thread-2569::INFO::2012-12-09 13:01:59,575::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2569::DEBUG::2012-12-09 13:01:59,575::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6717fdc6-ce8b-4c22-a692-65ecf60e4fa9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2569::DEBUG::2012-12-09 13:01:59,575::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2569::DEBUG::2012-12-09 13:01:59,575::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2569::DEBUG::2012-12-09 13:01:59,576::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6717fdc6-ce8b-4c22-a692-65ecf60e4fa9`::Granted request
>Thread-2569::DEBUG::2012-12-09 13:01:59,576::task::817::TaskManager.Task::(resourceAcquired) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2569::DEBUG::2012-12-09 13:01:59,576::task::978::TaskManager.Task::(_decref) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::ref 1 aborting False
>Thread-2569::DEBUG::2012-12-09 13:01:59,577::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2569::DEBUG::2012-12-09 13:01:59,578::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2569::DEBUG::2012-12-09 13:01:59,676::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2569::WARNING::2012-12-09 13:01:59,677::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2569::DEBUG::2012-12-09 13:01:59,677::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2569::WARNING::2012-12-09 13:01:59,678::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2569::INFO::2012-12-09 13:01:59,678::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2569::DEBUG::2012-12-09 13:01:59,678::task::1172::TaskManager.Task::(prepare) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2569::DEBUG::2012-12-09 13:01:59,679::task::588::TaskManager.Task::(_updateState) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::moving from state preparing -> state finished
>Thread-2569::DEBUG::2012-12-09 13:01:59,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2569::DEBUG::2012-12-09 13:01:59,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2569::DEBUG::2012-12-09 13:01:59,679::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2569::DEBUG::2012-12-09 13:01:59,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2569::DEBUG::2012-12-09 13:01:59,680::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2569::DEBUG::2012-12-09 13:01:59,680::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2569::DEBUG::2012-12-09 13:01:59,680::task::978::TaskManager.Task::(_decref) Task=`64918dd1-8bae-4327-8cde-895390d54ae5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:00,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:00,566::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170429 s, 60.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:02:02,549::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:02:02,561::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000547294 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:02,573::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:02,615::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171024 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:04,622::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:04,663::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165542 s, 61.9 MB/s\n'; <rc> = 0
>Thread-2572::DEBUG::2012-12-09 13:02:04,883::task::588::TaskManager.Task::(_updateState) Task=`2f6690d1-1d8f-4ff4-b08f-90b9586537a2`::moving from state init -> state preparing
>Thread-2572::INFO::2012-12-09 13:02:04,883::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2572::INFO::2012-12-09 13:02:04,884::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129861831665', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-2572::DEBUG::2012-12-09 13:02:04,884::task::1172::TaskManager.Task::(prepare) Task=`2f6690d1-1d8f-4ff4-b08f-90b9586537a2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129861831665', 'lastCheck': '2.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.4', 'code': 358, 'valid': False}}
>Thread-2572::DEBUG::2012-12-09 13:02:04,884::task::588::TaskManager.Task::(_updateState) Task=`2f6690d1-1d8f-4ff4-b08f-90b9586537a2`::moving from state preparing -> state finished
>Thread-2572::DEBUG::2012-12-09 13:02:04,884::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2572::DEBUG::2012-12-09 13:02:04,884::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2572::DEBUG::2012-12-09 13:02:04,885::task::978::TaskManager.Task::(_decref) Task=`2f6690d1-1d8f-4ff4-b08f-90b9586537a2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:06,670::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:06,711::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163641 s, 62.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:08,438::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:08,438::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:08,537::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:08,538::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:08,538::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:08,539::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:02:08,719::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:08,760::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168912 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2576::DEBUG::2012-12-09 13:02:09,943::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2576::DEBUG::2012-12-09 13:02:09,944::task::588::TaskManager.Task::(_updateState) Task=`c77404c1-d903-4753-9706-a05297edafdc`::moving from state init -> state preparing
>Thread-2576::INFO::2012-12-09 13:02:09,944::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2576::INFO::2012-12-09 13:02:09,944::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2576::DEBUG::2012-12-09 13:02:09,945::task::1172::TaskManager.Task::(prepare) Task=`c77404c1-d903-4753-9706-a05297edafdc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2576::DEBUG::2012-12-09 13:02:09,945::task::588::TaskManager.Task::(_updateState) Task=`c77404c1-d903-4753-9706-a05297edafdc`::moving from state preparing -> state finished
>Thread-2576::DEBUG::2012-12-09 13:02:09,945::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2576::DEBUG::2012-12-09 13:02:09,945::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2576::DEBUG::2012-12-09 13:02:09,945::task::978::TaskManager.Task::(_decref) Task=`c77404c1-d903-4753-9706-a05297edafdc`::ref 0 aborting False
>Thread-2577::DEBUG::2012-12-09 13:02:09,960::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2577::DEBUG::2012-12-09 13:02:09,961::task::588::TaskManager.Task::(_updateState) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::moving from state init -> state preparing
>Thread-2577::INFO::2012-12-09 13:02:09,961::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2577::DEBUG::2012-12-09 13:02:09,961::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0326c3c5-ab9c-431c-9fda-2d2b133b80df`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2577::DEBUG::2012-12-09 13:02:09,961::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2577::DEBUG::2012-12-09 13:02:09,962::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2577::DEBUG::2012-12-09 13:02:09,962::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0326c3c5-ab9c-431c-9fda-2d2b133b80df`::Granted request
>Thread-2577::DEBUG::2012-12-09 13:02:09,962::task::817::TaskManager.Task::(resourceAcquired) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2577::DEBUG::2012-12-09 13:02:09,962::task::978::TaskManager.Task::(_decref) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::ref 1 aborting False
>Thread-2577::DEBUG::2012-12-09 13:02:09,963::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2577::DEBUG::2012-12-09 13:02:09,964::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2577::DEBUG::2012-12-09 13:02:10,061::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2577::WARNING::2012-12-09 13:02:10,062::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2577::DEBUG::2012-12-09 13:02:10,062::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2577::WARNING::2012-12-09 13:02:10,063::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2577::INFO::2012-12-09 13:02:10,063::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2577::DEBUG::2012-12-09 13:02:10,063::task::1172::TaskManager.Task::(prepare) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2577::DEBUG::2012-12-09 13:02:10,064::task::588::TaskManager.Task::(_updateState) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::moving from state preparing -> state finished
>Thread-2577::DEBUG::2012-12-09 13:02:10,064::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2577::DEBUG::2012-12-09 13:02:10,064::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2577::DEBUG::2012-12-09 13:02:10,064::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2577::DEBUG::2012-12-09 13:02:10,065::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2577::DEBUG::2012-12-09 13:02:10,065::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2577::DEBUG::2012-12-09 13:02:10,065::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2577::DEBUG::2012-12-09 13:02:10,065::task::978::TaskManager.Task::(_decref) Task=`fdd584f5-f853-4e27-95e9-82d35336d522`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:10,767::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:10,807::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165141 s, 62.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:02:12,566::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:02:12,577::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00052867 s, 7.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:12,814::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:12,858::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192327 s, 53.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:14,865::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:14,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161689 s, 63.3 MB/s\n'; <rc> = 0
>Thread-2580::DEBUG::2012-12-09 13:02:15,530::task::588::TaskManager.Task::(_updateState) Task=`54278f47-bc7b-4c29-adc0-9ef4b097252a`::moving from state init -> state preparing
>Thread-2580::INFO::2012-12-09 13:02:15,530::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2580::INFO::2012-12-09 13:02:15,531::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118629932404', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.0', 'code': 358, 'valid': False}}
>Thread-2580::DEBUG::2012-12-09 13:02:15,531::task::1172::TaskManager.Task::(prepare) Task=`54278f47-bc7b-4c29-adc0-9ef4b097252a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118629932404', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.0', 'code': 358, 'valid': False}}
>Thread-2580::DEBUG::2012-12-09 13:02:15,531::task::588::TaskManager.Task::(_updateState) Task=`54278f47-bc7b-4c29-adc0-9ef4b097252a`::moving from state preparing -> state finished
>Thread-2580::DEBUG::2012-12-09 13:02:15,531::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2580::DEBUG::2012-12-09 13:02:15,531::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2580::DEBUG::2012-12-09 13:02:15,532::task::978::TaskManager.Task::(_decref) Task=`54278f47-bc7b-4c29-adc0-9ef4b097252a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:16,911::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:16,951::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162203 s, 63.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:18,540::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:18,540::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:18,644::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:18,646::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:18,646::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:18,646::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:02:18,958::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:18,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168423 s, 60.8 MB/s\n'; <rc> = 0
>Thread-2584::DEBUG::2012-12-09 13:02:20,451::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2584::DEBUG::2012-12-09 13:02:20,451::task::588::TaskManager.Task::(_updateState) Task=`7d949d28-5393-4917-a70a-5d5d30c63fa6`::moving from state init -> state preparing
>Thread-2584::INFO::2012-12-09 13:02:20,451::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2584::INFO::2012-12-09 13:02:20,452::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2584::DEBUG::2012-12-09 13:02:20,452::task::1172::TaskManager.Task::(prepare) Task=`7d949d28-5393-4917-a70a-5d5d30c63fa6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2584::DEBUG::2012-12-09 13:02:20,452::task::588::TaskManager.Task::(_updateState) Task=`7d949d28-5393-4917-a70a-5d5d30c63fa6`::moving from state preparing -> state finished
>Thread-2584::DEBUG::2012-12-09 13:02:20,452::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2584::DEBUG::2012-12-09 13:02:20,453::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2584::DEBUG::2012-12-09 13:02:20,453::task::978::TaskManager.Task::(_decref) Task=`7d949d28-5393-4917-a70a-5d5d30c63fa6`::ref 0 aborting False
>Thread-2585::DEBUG::2012-12-09 13:02:20,468::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2585::DEBUG::2012-12-09 13:02:20,469::task::588::TaskManager.Task::(_updateState) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::moving from state init -> state preparing
>Thread-2585::INFO::2012-12-09 13:02:20,469::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2585::DEBUG::2012-12-09 13:02:20,469::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e51aa710-6016-453a-992e-ba78e6ed72ac`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2585::DEBUG::2012-12-09 13:02:20,469::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2585::DEBUG::2012-12-09 13:02:20,470::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2585::DEBUG::2012-12-09 13:02:20,470::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e51aa710-6016-453a-992e-ba78e6ed72ac`::Granted request
>Thread-2585::DEBUG::2012-12-09 13:02:20,470::task::817::TaskManager.Task::(resourceAcquired) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2585::DEBUG::2012-12-09 13:02:20,470::task::978::TaskManager.Task::(_decref) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::ref 1 aborting False
>Thread-2585::DEBUG::2012-12-09 13:02:20,471::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2585::DEBUG::2012-12-09 13:02:20,472::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2585::DEBUG::2012-12-09 13:02:20,572::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2585::WARNING::2012-12-09 13:02:20,573::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2585::DEBUG::2012-12-09 13:02:20,573::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2585::WARNING::2012-12-09 13:02:20,574::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2585::INFO::2012-12-09 13:02:20,574::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2585::DEBUG::2012-12-09 13:02:20,574::task::1172::TaskManager.Task::(prepare) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2585::DEBUG::2012-12-09 13:02:20,575::task::588::TaskManager.Task::(_updateState) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::moving from state preparing -> state finished
>Thread-2585::DEBUG::2012-12-09 13:02:20,575::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2585::DEBUG::2012-12-09 13:02:20,575::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2585::DEBUG::2012-12-09 13:02:20,575::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2585::DEBUG::2012-12-09 13:02:20,575::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2585::DEBUG::2012-12-09 13:02:20,576::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2585::DEBUG::2012-12-09 13:02:20,576::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2585::DEBUG::2012-12-09 13:02:20,576::task::978::TaskManager.Task::(_decref) Task=`c3489500-1a6e-4c5d-8941-66226a350f49`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:21,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:21,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258912 s, 39.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:02:22,580::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:02:22,588::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000426955 s, 9.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:23,068::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:23,109::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169657 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:25,116::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:25,156::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168022 s, 60.9 MB/s\n'; <rc> = 0
>Thread-2588::DEBUG::2012-12-09 13:02:26,200::task::588::TaskManager.Task::(_updateState) Task=`560ea137-84d0-4c19-a7c3-e46f622250ef`::moving from state init -> state preparing
>Thread-2588::INFO::2012-12-09 13:02:26,201::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2588::INFO::2012-12-09 13:02:26,201::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00820684432983', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2588::DEBUG::2012-12-09 13:02:26,201::task::1172::TaskManager.Task::(prepare) Task=`560ea137-84d0-4c19-a7c3-e46f622250ef`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00820684432983', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.6', 'code': 358, 'valid': False}}
>Thread-2588::DEBUG::2012-12-09 13:02:26,202::task::588::TaskManager.Task::(_updateState) Task=`560ea137-84d0-4c19-a7c3-e46f622250ef`::moving from state preparing -> state finished
>Thread-2588::DEBUG::2012-12-09 13:02:26,202::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2588::DEBUG::2012-12-09 13:02:26,202::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2588::DEBUG::2012-12-09 13:02:26,202::task::978::TaskManager.Task::(_decref) Task=`560ea137-84d0-4c19-a7c3-e46f622250ef`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:27,163::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:27,207::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205911 s, 49.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:28,647::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:28,648::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:28,754::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:28,755::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:28,755::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:28,756::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:02:29,214::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:29,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169912 s, 60.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:31,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:31,301::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167934 s, 61.0 MB/s\n'; <rc> = 0
>Thread-2592::DEBUG::2012-12-09 13:02:32,226::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2592::DEBUG::2012-12-09 13:02:32,226::task::588::TaskManager.Task::(_updateState) Task=`40be251a-2e2a-4c18-8b73-b67be0f6a81e`::moving from state init -> state preparing
>Thread-2592::INFO::2012-12-09 13:02:32,226::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2592::INFO::2012-12-09 13:02:32,227::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2592::DEBUG::2012-12-09 13:02:32,227::task::1172::TaskManager.Task::(prepare) Task=`40be251a-2e2a-4c18-8b73-b67be0f6a81e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2592::DEBUG::2012-12-09 13:02:32,227::task::588::TaskManager.Task::(_updateState) Task=`40be251a-2e2a-4c18-8b73-b67be0f6a81e`::moving from state preparing -> state finished
>Thread-2592::DEBUG::2012-12-09 13:02:32,227::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2592::DEBUG::2012-12-09 13:02:32,227::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2592::DEBUG::2012-12-09 13:02:32,228::task::978::TaskManager.Task::(_decref) Task=`40be251a-2e2a-4c18-8b73-b67be0f6a81e`::ref 0 aborting False
>Thread-2593::DEBUG::2012-12-09 13:02:32,243::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2593::DEBUG::2012-12-09 13:02:32,243::task::588::TaskManager.Task::(_updateState) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::moving from state init -> state preparing
>Thread-2593::INFO::2012-12-09 13:02:32,243::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2593::DEBUG::2012-12-09 13:02:32,244::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`50457145-ff0a-4502-93ad-1fbdb8b1b967`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2593::DEBUG::2012-12-09 13:02:32,244::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2593::DEBUG::2012-12-09 13:02:32,244::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2593::DEBUG::2012-12-09 13:02:32,245::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`50457145-ff0a-4502-93ad-1fbdb8b1b967`::Granted request
>Thread-2593::DEBUG::2012-12-09 13:02:32,245::task::817::TaskManager.Task::(resourceAcquired) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2593::DEBUG::2012-12-09 13:02:32,245::task::978::TaskManager.Task::(_decref) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::ref 1 aborting False
>Thread-2593::DEBUG::2012-12-09 13:02:32,246::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2593::DEBUG::2012-12-09 13:02:32,246::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2593::DEBUG::2012-12-09 13:02:32,344::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2593::WARNING::2012-12-09 13:02:32,345::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2593::DEBUG::2012-12-09 13:02:32,346::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2593::WARNING::2012-12-09 13:02:32,346::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2593::INFO::2012-12-09 13:02:32,347::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2593::DEBUG::2012-12-09 13:02:32,347::task::1172::TaskManager.Task::(prepare) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2593::DEBUG::2012-12-09 13:02:32,347::task::588::TaskManager.Task::(_updateState) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::moving from state preparing -> state finished
>Thread-2593::DEBUG::2012-12-09 13:02:32,347::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2593::DEBUG::2012-12-09 13:02:32,348::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2593::DEBUG::2012-12-09 13:02:32,349::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2593::DEBUG::2012-12-09 13:02:32,349::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2593::DEBUG::2012-12-09 13:02:32,349::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2593::DEBUG::2012-12-09 13:02:32,350::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2593::DEBUG::2012-12-09 13:02:32,350::task::978::TaskManager.Task::(_decref) Task=`1eed7478-c6e2-47ce-9d4e-4adbebed4f33`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:02:32,591::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:02:32,599::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000502711 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:33,308::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:33,349::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164965 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:35,356::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:35,398::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018311 s, 55.9 MB/s\n'; <rc> = 0
>Thread-2596::DEBUG::2012-12-09 13:02:36,407::task::588::TaskManager.Task::(_updateState) Task=`d3f8b815-9ba8-4bab-9c9c-f8ce51a406b8`::moving from state init -> state preparing
>Thread-2596::INFO::2012-12-09 13:02:36,407::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2596::INFO::2012-12-09 13:02:36,407::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00863289833069', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2596::DEBUG::2012-12-09 13:02:36,408::task::1172::TaskManager.Task::(prepare) Task=`d3f8b815-9ba8-4bab-9c9c-f8ce51a406b8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00863289833069', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2596::DEBUG::2012-12-09 13:02:36,408::task::588::TaskManager.Task::(_updateState) Task=`d3f8b815-9ba8-4bab-9c9c-f8ce51a406b8`::moving from state preparing -> state finished
>Thread-2596::DEBUG::2012-12-09 13:02:36,408::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2596::DEBUG::2012-12-09 13:02:36,408::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2596::DEBUG::2012-12-09 13:02:36,408::task::978::TaskManager.Task::(_decref) Task=`d3f8b815-9ba8-4bab-9c9c-f8ce51a406b8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:37,405::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:37,445::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017473 s, 58.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:38,757::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:38,758::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:38,857::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:38,858::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:38,859::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:38,859::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:02:39,452::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:39,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273189 s, 37.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:41,513::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:41,556::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174287 s, 58.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:02:42,603::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-2601::DEBUG::2012-12-09 13:02:42,610::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2601::DEBUG::2012-12-09 13:02:42,612::task::588::TaskManager.Task::(_updateState) Task=`d6885cd7-38c9-4a56-a8b8-10851464b1b6`::moving from state init -> state preparing
>Thread-2601::INFO::2012-12-09 13:02:42,614::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2601::INFO::2012-12-09 13:02:42,616::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2601::DEBUG::2012-12-09 13:02:42,617::task::1172::TaskManager.Task::(prepare) Task=`d6885cd7-38c9-4a56-a8b8-10851464b1b6`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2601::DEBUG::2012-12-09 13:02:42,619::task::588::TaskManager.Task::(_updateState) Task=`d6885cd7-38c9-4a56-a8b8-10851464b1b6`::moving from state preparing -> state finished
>Thread-50::DEBUG::2012-12-09 13:02:42,620::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000354054 s, 11.6 MB/s\n'; <rc> = 0
>Thread-2601::DEBUG::2012-12-09 13:02:42,620::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2601::DEBUG::2012-12-09 13:02:42,621::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2601::DEBUG::2012-12-09 13:02:42,622::task::978::TaskManager.Task::(_decref) Task=`d6885cd7-38c9-4a56-a8b8-10851464b1b6`::ref 0 aborting False
>Thread-2602::DEBUG::2012-12-09 13:02:42,642::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2602::DEBUG::2012-12-09 13:02:42,643::task::588::TaskManager.Task::(_updateState) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::moving from state init -> state preparing
>Thread-2602::INFO::2012-12-09 13:02:42,643::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2602::DEBUG::2012-12-09 13:02:42,643::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`78831098-6ee8-4b26-a0b0-0547620d1373`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2602::DEBUG::2012-12-09 13:02:42,643::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2602::DEBUG::2012-12-09 13:02:42,644::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2602::DEBUG::2012-12-09 13:02:42,644::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`78831098-6ee8-4b26-a0b0-0547620d1373`::Granted request
>Thread-2602::DEBUG::2012-12-09 13:02:42,644::task::817::TaskManager.Task::(resourceAcquired) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2602::DEBUG::2012-12-09 13:02:42,644::task::978::TaskManager.Task::(_decref) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::ref 1 aborting False
>Thread-2602::DEBUG::2012-12-09 13:02:42,645::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2602::DEBUG::2012-12-09 13:02:42,646::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2602::DEBUG::2012-12-09 13:02:42,764::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2602::WARNING::2012-12-09 13:02:42,765::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2602::DEBUG::2012-12-09 13:02:42,765::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2602::WARNING::2012-12-09 13:02:42,766::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2602::INFO::2012-12-09 13:02:42,766::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2602::DEBUG::2012-12-09 13:02:42,766::task::1172::TaskManager.Task::(prepare) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2602::DEBUG::2012-12-09 13:02:42,767::task::588::TaskManager.Task::(_updateState) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::moving from state preparing -> state finished
>Thread-2602::DEBUG::2012-12-09 13:02:42,767::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2602::DEBUG::2012-12-09 13:02:42,767::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2602::DEBUG::2012-12-09 13:02:42,767::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2602::DEBUG::2012-12-09 13:02:42,768::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2602::DEBUG::2012-12-09 13:02:42,768::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2602::DEBUG::2012-12-09 13:02:42,768::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2602::DEBUG::2012-12-09 13:02:42,768::task::978::TaskManager.Task::(_decref) Task=`22128597-6188-4b80-8eaf-5cae1dead500`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:43,563::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:43,616::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0252758 s, 40.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:45,623::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:45,672::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180121 s, 56.9 MB/s\n'; <rc> = 0
>Thread-2604::DEBUG::2012-12-09 13:02:46,598::task::588::TaskManager.Task::(_updateState) Task=`6c613001-b6d6-4332-b091-3c79f6be7296`::moving from state init -> state preparing
>Thread-2604::INFO::2012-12-09 13:02:46,598::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2604::INFO::2012-12-09 13:02:46,599::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0178520679474', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2604::DEBUG::2012-12-09 13:02:46,599::task::1172::TaskManager.Task::(prepare) Task=`6c613001-b6d6-4332-b091-3c79f6be7296`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0178520679474', 'lastCheck': '4.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.7', 'code': 358, 'valid': False}}
>Thread-2604::DEBUG::2012-12-09 13:02:46,599::task::588::TaskManager.Task::(_updateState) Task=`6c613001-b6d6-4332-b091-3c79f6be7296`::moving from state preparing -> state finished
>Thread-2604::DEBUG::2012-12-09 13:02:46,599::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2604::DEBUG::2012-12-09 13:02:46,600::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2604::DEBUG::2012-12-09 13:02:46,600::task::978::TaskManager.Task::(_decref) Task=`6c613001-b6d6-4332-b091-3c79f6be7296`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:47,680::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:47,726::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194901 s, 52.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:48,860::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:48,861::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:48,962::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:48,963::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:48,964::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:48,964::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:02:49,733::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:49,799::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262629 s, 39.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:51,809::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:51,849::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169587 s, 60.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:02:52,629::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:02:52,641::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00061798 s, 6.6 MB/s\n'; <rc> = 0
>Thread-2608::DEBUG::2012-12-09 13:02:52,821::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2608::DEBUG::2012-12-09 13:02:52,821::task::588::TaskManager.Task::(_updateState) Task=`91e1ebba-ed84-4533-956b-fd2f868925d7`::moving from state init -> state preparing
>Thread-2608::INFO::2012-12-09 13:02:52,822::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2608::INFO::2012-12-09 13:02:52,822::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2608::DEBUG::2012-12-09 13:02:52,822::task::1172::TaskManager.Task::(prepare) Task=`91e1ebba-ed84-4533-956b-fd2f868925d7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2608::DEBUG::2012-12-09 13:02:52,822::task::588::TaskManager.Task::(_updateState) Task=`91e1ebba-ed84-4533-956b-fd2f868925d7`::moving from state preparing -> state finished
>Thread-2608::DEBUG::2012-12-09 13:02:52,822::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2608::DEBUG::2012-12-09 13:02:52,823::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2608::DEBUG::2012-12-09 13:02:52,823::task::978::TaskManager.Task::(_decref) Task=`91e1ebba-ed84-4533-956b-fd2f868925d7`::ref 0 aborting False
>Thread-2609::DEBUG::2012-12-09 13:02:52,842::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2609::DEBUG::2012-12-09 13:02:52,842::task::588::TaskManager.Task::(_updateState) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::moving from state init -> state preparing
>Thread-2609::INFO::2012-12-09 13:02:52,843::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2609::DEBUG::2012-12-09 13:02:52,843::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8f21eb37-b97c-4a87-a1f7-853a9bffe9cc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2609::DEBUG::2012-12-09 13:02:52,843::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2609::DEBUG::2012-12-09 13:02:52,844::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2609::DEBUG::2012-12-09 13:02:52,845::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8f21eb37-b97c-4a87-a1f7-853a9bffe9cc`::Granted request
>Thread-2609::DEBUG::2012-12-09 13:02:52,847::task::817::TaskManager.Task::(resourceAcquired) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2609::DEBUG::2012-12-09 13:02:52,847::task::978::TaskManager.Task::(_decref) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::ref 1 aborting False
>Thread-2609::DEBUG::2012-12-09 13:02:52,848::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2609::DEBUG::2012-12-09 13:02:52,848::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2609::DEBUG::2012-12-09 13:02:52,947::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2609::WARNING::2012-12-09 13:02:52,948::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2609::DEBUG::2012-12-09 13:02:52,948::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2609::WARNING::2012-12-09 13:02:52,949::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2609::INFO::2012-12-09 13:02:52,949::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2609::DEBUG::2012-12-09 13:02:52,949::task::1172::TaskManager.Task::(prepare) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2609::DEBUG::2012-12-09 13:02:52,950::task::588::TaskManager.Task::(_updateState) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::moving from state preparing -> state finished
>Thread-2609::DEBUG::2012-12-09 13:02:52,950::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2609::DEBUG::2012-12-09 13:02:52,950::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2609::DEBUG::2012-12-09 13:02:52,950::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2609::DEBUG::2012-12-09 13:02:52,951::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2609::DEBUG::2012-12-09 13:02:52,951::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2609::DEBUG::2012-12-09 13:02:52,951::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2609::DEBUG::2012-12-09 13:02:52,951::task::978::TaskManager.Task::(_decref) Task=`bf1b8021-939d-48b2-9569-963117ac93e1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:53,856::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:53,897::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172903 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:02:55,904::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:55,954::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026278 s, 39.0 MB/s\n'; <rc> = 0
>Thread-2612::DEBUG::2012-12-09 13:02:56,896::task::588::TaskManager.Task::(_updateState) Task=`50f7a9d3-ece3-4965-85b6-051bccd6f36c`::moving from state init -> state preparing
>Thread-2612::INFO::2012-12-09 13:02:56,897::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2612::INFO::2012-12-09 13:02:56,897::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128228664398', 'lastCheck': '4.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-2612::DEBUG::2012-12-09 13:02:56,897::task::1172::TaskManager.Task::(prepare) Task=`50f7a9d3-ece3-4965-85b6-051bccd6f36c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128228664398', 'lastCheck': '4.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-2612::DEBUG::2012-12-09 13:02:56,898::task::588::TaskManager.Task::(_updateState) Task=`50f7a9d3-ece3-4965-85b6-051bccd6f36c`::moving from state preparing -> state finished
>Thread-2612::DEBUG::2012-12-09 13:02:56,898::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2612::DEBUG::2012-12-09 13:02:56,898::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2612::DEBUG::2012-12-09 13:02:56,898::task::978::TaskManager.Task::(_decref) Task=`50f7a9d3-ece3-4965-85b6-051bccd6f36c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:02:57,961::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:02:58,003::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179347 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:02:58,965::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:02:58,966::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:02:59,063::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:02:59,064::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:02:59,064::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:02:59,065::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:00,010::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:00,051::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169321 s, 60.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:02,058::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:02,121::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262744 s, 39.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:02,645::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:02,653::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000529798 s, 7.7 MB/s\n'; <rc> = 0
>Thread-2616::DEBUG::2012-12-09 13:03:03,097::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2616::DEBUG::2012-12-09 13:03:03,097::task::588::TaskManager.Task::(_updateState) Task=`727c3e4c-f352-4032-94df-c6315b184c56`::moving from state init -> state preparing
>Thread-2616::INFO::2012-12-09 13:03:03,098::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2616::INFO::2012-12-09 13:03:03,098::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2616::DEBUG::2012-12-09 13:03:03,098::task::1172::TaskManager.Task::(prepare) Task=`727c3e4c-f352-4032-94df-c6315b184c56`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2616::DEBUG::2012-12-09 13:03:03,098::task::588::TaskManager.Task::(_updateState) Task=`727c3e4c-f352-4032-94df-c6315b184c56`::moving from state preparing -> state finished
>Thread-2616::DEBUG::2012-12-09 13:03:03,099::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2616::DEBUG::2012-12-09 13:03:03,099::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2616::DEBUG::2012-12-09 13:03:03,099::task::978::TaskManager.Task::(_decref) Task=`727c3e4c-f352-4032-94df-c6315b184c56`::ref 0 aborting False
>Thread-2617::DEBUG::2012-12-09 13:03:03,114::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2617::DEBUG::2012-12-09 13:03:03,115::task::588::TaskManager.Task::(_updateState) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::moving from state init -> state preparing
>Thread-2617::INFO::2012-12-09 13:03:03,115::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2617::DEBUG::2012-12-09 13:03:03,116::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ac5b6170-d9c7-4dd6-8ed6-0a189850d392`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2617::DEBUG::2012-12-09 13:03:03,116::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2617::DEBUG::2012-12-09 13:03:03,116::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2617::DEBUG::2012-12-09 13:03:03,116::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ac5b6170-d9c7-4dd6-8ed6-0a189850d392`::Granted request
>Thread-2617::DEBUG::2012-12-09 13:03:03,117::task::817::TaskManager.Task::(resourceAcquired) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2617::DEBUG::2012-12-09 13:03:03,117::task::978::TaskManager.Task::(_decref) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::ref 1 aborting False
>Thread-2617::DEBUG::2012-12-09 13:03:03,117::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2617::DEBUG::2012-12-09 13:03:03,118::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2617::DEBUG::2012-12-09 13:03:03,224::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2617::WARNING::2012-12-09 13:03:03,226::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2617::DEBUG::2012-12-09 13:03:03,226::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2617::WARNING::2012-12-09 13:03:03,227::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2617::INFO::2012-12-09 13:03:03,227::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2617::DEBUG::2012-12-09 13:03:03,227::task::1172::TaskManager.Task::(prepare) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2617::DEBUG::2012-12-09 13:03:03,228::task::588::TaskManager.Task::(_updateState) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::moving from state preparing -> state finished
>Thread-2617::DEBUG::2012-12-09 13:03:03,228::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2617::DEBUG::2012-12-09 13:03:03,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2617::DEBUG::2012-12-09 13:03:03,228::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2617::DEBUG::2012-12-09 13:03:03,228::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2617::DEBUG::2012-12-09 13:03:03,229::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2617::DEBUG::2012-12-09 13:03:03,229::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2617::DEBUG::2012-12-09 13:03:03,229::task::978::TaskManager.Task::(_decref) Task=`c9e7dca0-ba33-4f5d-8c02-69ed3d10c20b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:04,127::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:04,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160373 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:06,175::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:06,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170046 s, 60.2 MB/s\n'; <rc> = 0
>Thread-2620::DEBUG::2012-12-09 13:03:07,241::task::588::TaskManager.Task::(_updateState) Task=`8b4989a4-988c-4c4a-b2df-c68343bc246f`::moving from state init -> state preparing
>Thread-2620::INFO::2012-12-09 13:03:07,241::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2620::INFO::2012-12-09 13:03:07,241::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00887203216553', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-2620::DEBUG::2012-12-09 13:03:07,242::task::1172::TaskManager.Task::(prepare) Task=`8b4989a4-988c-4c4a-b2df-c68343bc246f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00887203216553', 'lastCheck': '4.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-2620::DEBUG::2012-12-09 13:03:07,242::task::588::TaskManager.Task::(_updateState) Task=`8b4989a4-988c-4c4a-b2df-c68343bc246f`::moving from state preparing -> state finished
>Thread-2620::DEBUG::2012-12-09 13:03:07,242::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2620::DEBUG::2012-12-09 13:03:07,242::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2620::DEBUG::2012-12-09 13:03:07,242::task::978::TaskManager.Task::(_decref) Task=`8b4989a4-988c-4c4a-b2df-c68343bc246f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:08,223::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:08,262::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174323 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:09,066::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:09,067::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:09,169::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:09,171::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:09,171::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:09,172::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:10,269::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:10,308::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161912 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:12,315::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:12,353::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162447 s, 63.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:12,656::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:12,664::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000518646 s, 7.9 MB/s\n'; <rc> = 0
>Thread-2624::DEBUG::2012-12-09 13:03:13,364::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2624::DEBUG::2012-12-09 13:03:13,365::task::588::TaskManager.Task::(_updateState) Task=`d4c79541-1b19-4c91-b246-982af96dcd16`::moving from state init -> state preparing
>Thread-2624::INFO::2012-12-09 13:03:13,365::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2624::INFO::2012-12-09 13:03:13,366::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2624::DEBUG::2012-12-09 13:03:13,366::task::1172::TaskManager.Task::(prepare) Task=`d4c79541-1b19-4c91-b246-982af96dcd16`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2624::DEBUG::2012-12-09 13:03:13,366::task::588::TaskManager.Task::(_updateState) Task=`d4c79541-1b19-4c91-b246-982af96dcd16`::moving from state preparing -> state finished
>Thread-2624::DEBUG::2012-12-09 13:03:13,366::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2624::DEBUG::2012-12-09 13:03:13,366::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2624::DEBUG::2012-12-09 13:03:13,366::task::978::TaskManager.Task::(_decref) Task=`d4c79541-1b19-4c91-b246-982af96dcd16`::ref 0 aborting False
>Thread-2625::DEBUG::2012-12-09 13:03:13,380::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2625::DEBUG::2012-12-09 13:03:13,380::task::588::TaskManager.Task::(_updateState) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::moving from state init -> state preparing
>Thread-2625::INFO::2012-12-09 13:03:13,380::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2625::DEBUG::2012-12-09 13:03:13,381::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1da281f3-cee0-4812-bb72-b781608d153c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2625::DEBUG::2012-12-09 13:03:13,381::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2625::DEBUG::2012-12-09 13:03:13,381::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2625::DEBUG::2012-12-09 13:03:13,381::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1da281f3-cee0-4812-bb72-b781608d153c`::Granted request
>Thread-2625::DEBUG::2012-12-09 13:03:13,382::task::817::TaskManager.Task::(resourceAcquired) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2625::DEBUG::2012-12-09 13:03:13,382::task::978::TaskManager.Task::(_decref) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::ref 1 aborting False
>Thread-2625::DEBUG::2012-12-09 13:03:13,383::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2625::DEBUG::2012-12-09 13:03:13,383::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2625::DEBUG::2012-12-09 13:03:13,483::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2625::WARNING::2012-12-09 13:03:13,484::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2625::DEBUG::2012-12-09 13:03:13,485::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2625::WARNING::2012-12-09 13:03:13,485::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2625::INFO::2012-12-09 13:03:13,485::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2625::DEBUG::2012-12-09 13:03:13,486::task::1172::TaskManager.Task::(prepare) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2625::DEBUG::2012-12-09 13:03:13,486::task::588::TaskManager.Task::(_updateState) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::moving from state preparing -> state finished
>Thread-2625::DEBUG::2012-12-09 13:03:13,486::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2625::DEBUG::2012-12-09 13:03:13,486::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2625::DEBUG::2012-12-09 13:03:13,486::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2625::DEBUG::2012-12-09 13:03:13,487::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2625::DEBUG::2012-12-09 13:03:13,487::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2625::DEBUG::2012-12-09 13:03:13,487::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2625::DEBUG::2012-12-09 13:03:13,487::task::978::TaskManager.Task::(_decref) Task=`66417456-7dc7-46d1-9e47-81c0ea99c53a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:14,360::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:14,401::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172125 s, 59.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:16,408::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:16,448::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168982 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2628::DEBUG::2012-12-09 13:03:17,523::task::588::TaskManager.Task::(_updateState) Task=`d9aa767a-62d4-4d8d-9fb5-5e6ecd68ff2c`::moving from state init -> state preparing
>Thread-2628::INFO::2012-12-09 13:03:17,523::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2628::INFO::2012-12-09 13:03:17,523::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850701332092', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-2628::DEBUG::2012-12-09 13:03:17,524::task::1172::TaskManager.Task::(prepare) Task=`d9aa767a-62d4-4d8d-9fb5-5e6ecd68ff2c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850701332092', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-2628::DEBUG::2012-12-09 13:03:17,524::task::588::TaskManager.Task::(_updateState) Task=`d9aa767a-62d4-4d8d-9fb5-5e6ecd68ff2c`::moving from state preparing -> state finished
>Thread-2628::DEBUG::2012-12-09 13:03:17,524::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2628::DEBUG::2012-12-09 13:03:17,524::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2628::DEBUG::2012-12-09 13:03:17,524::task::978::TaskManager.Task::(_decref) Task=`d9aa767a-62d4-4d8d-9fb5-5e6ecd68ff2c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:18,455::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:18,495::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166007 s, 61.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:19,172::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:19,173::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:19,269::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:19,271::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:19,271::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:19,271::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:20,502::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:20,542::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164462 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:22,549::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:22,590::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016768 s, 61.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:22,667::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:22,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00042458 s, 9.6 MB/s\n'; <rc> = 0
>Thread-2632::DEBUG::2012-12-09 13:03:23,880::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2632::DEBUG::2012-12-09 13:03:23,881::task::588::TaskManager.Task::(_updateState) Task=`31c373a4-5e73-4c31-bd45-bcc435e31368`::moving from state init -> state preparing
>Thread-2632::INFO::2012-12-09 13:03:23,881::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2632::INFO::2012-12-09 13:03:23,882::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2632::DEBUG::2012-12-09 13:03:23,882::task::1172::TaskManager.Task::(prepare) Task=`31c373a4-5e73-4c31-bd45-bcc435e31368`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2632::DEBUG::2012-12-09 13:03:23,882::task::588::TaskManager.Task::(_updateState) Task=`31c373a4-5e73-4c31-bd45-bcc435e31368`::moving from state preparing -> state finished
>Thread-2632::DEBUG::2012-12-09 13:03:23,883::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2632::DEBUG::2012-12-09 13:03:23,883::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2632::DEBUG::2012-12-09 13:03:23,883::task::978::TaskManager.Task::(_decref) Task=`31c373a4-5e73-4c31-bd45-bcc435e31368`::ref 0 aborting False
>Thread-2633::DEBUG::2012-12-09 13:03:23,898::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2633::DEBUG::2012-12-09 13:03:23,898::task::588::TaskManager.Task::(_updateState) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::moving from state init -> state preparing
>Thread-2633::INFO::2012-12-09 13:03:23,899::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2633::DEBUG::2012-12-09 13:03:23,899::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3d7d678a-ba46-4983-96ce-fcbbb3baa7dc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2633::DEBUG::2012-12-09 13:03:23,899::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2633::DEBUG::2012-12-09 13:03:23,900::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2633::DEBUG::2012-12-09 13:03:23,900::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3d7d678a-ba46-4983-96ce-fcbbb3baa7dc`::Granted request
>Thread-2633::DEBUG::2012-12-09 13:03:23,900::task::817::TaskManager.Task::(resourceAcquired) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2633::DEBUG::2012-12-09 13:03:23,900::task::978::TaskManager.Task::(_decref) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::ref 1 aborting False
>Thread-2633::DEBUG::2012-12-09 13:03:23,901::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2633::DEBUG::2012-12-09 13:03:23,902::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2633::DEBUG::2012-12-09 13:03:24,007::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2633::WARNING::2012-12-09 13:03:24,008::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2633::DEBUG::2012-12-09 13:03:24,008::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2633::WARNING::2012-12-09 13:03:24,008::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2633::INFO::2012-12-09 13:03:24,009::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2633::DEBUG::2012-12-09 13:03:24,009::task::1172::TaskManager.Task::(prepare) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2633::DEBUG::2012-12-09 13:03:24,009::task::588::TaskManager.Task::(_updateState) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::moving from state preparing -> state finished
>Thread-2633::DEBUG::2012-12-09 13:03:24,009::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2633::DEBUG::2012-12-09 13:03:24,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2633::DEBUG::2012-12-09 13:03:24,010::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2633::DEBUG::2012-12-09 13:03:24,010::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2633::DEBUG::2012-12-09 13:03:24,010::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2633::DEBUG::2012-12-09 13:03:24,011::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2633::DEBUG::2012-12-09 13:03:24,011::task::978::TaskManager.Task::(_decref) Task=`b4b7b10e-28d5-47fb-8526-8285e4610276`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:24,597::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:24,638::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175019 s, 58.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:26,644::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:26,685::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166781 s, 61.4 MB/s\n'; <rc> = 0
>Thread-2636::DEBUG::2012-12-09 13:03:28,048::task::588::TaskManager.Task::(_updateState) Task=`41698aee-d3b8-4625-8dda-e1fac9a9c91b`::moving from state init -> state preparing
>Thread-2636::INFO::2012-12-09 13:03:28,048::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2636::INFO::2012-12-09 13:03:28,048::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00931000709534', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-2636::DEBUG::2012-12-09 13:03:28,049::task::1172::TaskManager.Task::(prepare) Task=`41698aee-d3b8-4625-8dda-e1fac9a9c91b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00931000709534', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-2636::DEBUG::2012-12-09 13:03:28,049::task::588::TaskManager.Task::(_updateState) Task=`41698aee-d3b8-4625-8dda-e1fac9a9c91b`::moving from state preparing -> state finished
>Thread-2636::DEBUG::2012-12-09 13:03:28,049::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2636::DEBUG::2012-12-09 13:03:28,049::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2636::DEBUG::2012-12-09 13:03:28,049::task::978::TaskManager.Task::(_decref) Task=`41698aee-d3b8-4625-8dda-e1fac9a9c91b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:28,692::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:28,736::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019135 s, 53.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:29,272::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:29,273::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:29,375::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:29,376::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:29,377::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:29,378::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:30,743::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:30,785::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174485 s, 58.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:32,679::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:32,690::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000567264 s, 7.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:32,792::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:32,833::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167472 s, 61.1 MB/s\n'; <rc> = 0
>Thread-2640::DEBUG::2012-12-09 13:03:34,378::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2640::DEBUG::2012-12-09 13:03:34,379::task::588::TaskManager.Task::(_updateState) Task=`a5e9da56-df60-43a0-ba4b-3a1175267037`::moving from state init -> state preparing
>Thread-2640::INFO::2012-12-09 13:03:34,379::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2640::INFO::2012-12-09 13:03:34,380::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2640::DEBUG::2012-12-09 13:03:34,380::task::1172::TaskManager.Task::(prepare) Task=`a5e9da56-df60-43a0-ba4b-3a1175267037`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2640::DEBUG::2012-12-09 13:03:34,380::task::588::TaskManager.Task::(_updateState) Task=`a5e9da56-df60-43a0-ba4b-3a1175267037`::moving from state preparing -> state finished
>Thread-2640::DEBUG::2012-12-09 13:03:34,380::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2640::DEBUG::2012-12-09 13:03:34,380::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2640::DEBUG::2012-12-09 13:03:34,381::task::978::TaskManager.Task::(_decref) Task=`a5e9da56-df60-43a0-ba4b-3a1175267037`::ref 0 aborting False
>Thread-2641::DEBUG::2012-12-09 13:03:34,393::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2641::DEBUG::2012-12-09 13:03:34,394::task::588::TaskManager.Task::(_updateState) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::moving from state init -> state preparing
>Thread-2641::INFO::2012-12-09 13:03:34,394::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2641::DEBUG::2012-12-09 13:03:34,394::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`964e1a36-6bd9-47d6-a66c-003c16f3f230`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2641::DEBUG::2012-12-09 13:03:34,394::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2641::DEBUG::2012-12-09 13:03:34,395::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2641::DEBUG::2012-12-09 13:03:34,395::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`964e1a36-6bd9-47d6-a66c-003c16f3f230`::Granted request
>Thread-2641::DEBUG::2012-12-09 13:03:34,395::task::817::TaskManager.Task::(resourceAcquired) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2641::DEBUG::2012-12-09 13:03:34,395::task::978::TaskManager.Task::(_decref) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::ref 1 aborting False
>Thread-2641::DEBUG::2012-12-09 13:03:34,396::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2641::DEBUG::2012-12-09 13:03:34,397::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2641::DEBUG::2012-12-09 13:03:34,502::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2641::WARNING::2012-12-09 13:03:34,504::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2641::DEBUG::2012-12-09 13:03:34,504::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2641::WARNING::2012-12-09 13:03:34,504::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2641::INFO::2012-12-09 13:03:34,505::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2641::DEBUG::2012-12-09 13:03:34,505::task::1172::TaskManager.Task::(prepare) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2641::DEBUG::2012-12-09 13:03:34,505::task::588::TaskManager.Task::(_updateState) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::moving from state preparing -> state finished
>Thread-2641::DEBUG::2012-12-09 13:03:34,505::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2641::DEBUG::2012-12-09 13:03:34,506::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2641::DEBUG::2012-12-09 13:03:34,506::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2641::DEBUG::2012-12-09 13:03:34,506::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2641::DEBUG::2012-12-09 13:03:34,506::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2641::DEBUG::2012-12-09 13:03:34,507::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2641::DEBUG::2012-12-09 13:03:34,507::task::978::TaskManager.Task::(_decref) Task=`8ad6024d-1228-4669-9106-5f3ba77eeba2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:34,839::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:34,881::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179587 s, 57.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:36,888::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:36,929::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178067 s, 57.5 MB/s\n'; <rc> = 0
>Thread-2644::DEBUG::2012-12-09 13:03:38,470::task::588::TaskManager.Task::(_updateState) Task=`9852a124-3e20-4dc3-92a5-ec349eea99c6`::moving from state init -> state preparing
>Thread-2644::INFO::2012-12-09 13:03:38,470::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2644::INFO::2012-12-09 13:03:38,471::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120060443878', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-2644::DEBUG::2012-12-09 13:03:38,471::task::1172::TaskManager.Task::(prepare) Task=`9852a124-3e20-4dc3-92a5-ec349eea99c6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120060443878', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.1', 'code': 358, 'valid': False}}
>Thread-2644::DEBUG::2012-12-09 13:03:38,471::task::588::TaskManager.Task::(_updateState) Task=`9852a124-3e20-4dc3-92a5-ec349eea99c6`::moving from state preparing -> state finished
>Thread-2644::DEBUG::2012-12-09 13:03:38,471::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2644::DEBUG::2012-12-09 13:03:38,472::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2644::DEBUG::2012-12-09 13:03:38,472::task::978::TaskManager.Task::(_decref) Task=`9852a124-3e20-4dc3-92a5-ec349eea99c6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:38,937::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:38,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170026 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:39,379::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:39,380::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:39,479::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:39,480::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:39,480::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:39,481::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:40,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:41,024::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016515 s, 62.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:42,693::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:42,705::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000475328 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:43,031::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:43,075::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162288 s, 63.1 MB/s\n'; <rc> = 0
>Thread-2648::DEBUG::2012-12-09 13:03:44,562::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2648::DEBUG::2012-12-09 13:03:44,562::task::588::TaskManager.Task::(_updateState) Task=`08445567-f099-4a36-82fa-ac811fd27222`::moving from state init -> state preparing
>Thread-2648::INFO::2012-12-09 13:03:44,562::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2648::INFO::2012-12-09 13:03:44,563::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2648::DEBUG::2012-12-09 13:03:44,563::task::1172::TaskManager.Task::(prepare) Task=`08445567-f099-4a36-82fa-ac811fd27222`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2648::DEBUG::2012-12-09 13:03:44,563::task::588::TaskManager.Task::(_updateState) Task=`08445567-f099-4a36-82fa-ac811fd27222`::moving from state preparing -> state finished
>Thread-2648::DEBUG::2012-12-09 13:03:44,563::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2648::DEBUG::2012-12-09 13:03:44,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2648::DEBUG::2012-12-09 13:03:44,564::task::978::TaskManager.Task::(_decref) Task=`08445567-f099-4a36-82fa-ac811fd27222`::ref 0 aborting False
>Thread-2649::DEBUG::2012-12-09 13:03:44,576::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2649::DEBUG::2012-12-09 13:03:44,577::task::588::TaskManager.Task::(_updateState) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::moving from state init -> state preparing
>Thread-2649::INFO::2012-12-09 13:03:44,577::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2649::DEBUG::2012-12-09 13:03:44,577::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d8dc867-b615-4a88-a3e9-07982f1199e7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2649::DEBUG::2012-12-09 13:03:44,577::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2649::DEBUG::2012-12-09 13:03:44,578::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2649::DEBUG::2012-12-09 13:03:44,578::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5d8dc867-b615-4a88-a3e9-07982f1199e7`::Granted request
>Thread-2649::DEBUG::2012-12-09 13:03:44,578::task::817::TaskManager.Task::(resourceAcquired) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2649::DEBUG::2012-12-09 13:03:44,578::task::978::TaskManager.Task::(_decref) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::ref 1 aborting False
>Thread-2649::DEBUG::2012-12-09 13:03:44,579::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2649::DEBUG::2012-12-09 13:03:44,579::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2649::DEBUG::2012-12-09 13:03:44,677::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2649::WARNING::2012-12-09 13:03:44,679::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2649::DEBUG::2012-12-09 13:03:44,679::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2649::WARNING::2012-12-09 13:03:44,679::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2649::INFO::2012-12-09 13:03:44,680::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2649::DEBUG::2012-12-09 13:03:44,680::task::1172::TaskManager.Task::(prepare) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2649::DEBUG::2012-12-09 13:03:44,680::task::588::TaskManager.Task::(_updateState) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::moving from state preparing -> state finished
>Thread-2649::DEBUG::2012-12-09 13:03:44,681::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2649::DEBUG::2012-12-09 13:03:44,681::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2649::DEBUG::2012-12-09 13:03:44,681::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2649::DEBUG::2012-12-09 13:03:44,681::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2649::DEBUG::2012-12-09 13:03:44,681::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2649::DEBUG::2012-12-09 13:03:44,682::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2649::DEBUG::2012-12-09 13:03:44,682::task::978::TaskManager.Task::(_decref) Task=`d0492ca4-5d54-41af-a75f-6593216579e9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:45,082::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:45,125::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019574 s, 52.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:47,132::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:47,173::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164742 s, 62.2 MB/s\n'; <rc> = 0
>Thread-2652::DEBUG::2012-12-09 13:03:48,751::task::588::TaskManager.Task::(_updateState) Task=`318ece2e-b699-473f-9a1e-8100defa918b`::moving from state init -> state preparing
>Thread-2652::INFO::2012-12-09 13:03:48,751::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2652::INFO::2012-12-09 13:03:48,752::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119590759277', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2652::DEBUG::2012-12-09 13:03:48,752::task::1172::TaskManager.Task::(prepare) Task=`318ece2e-b699-473f-9a1e-8100defa918b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119590759277', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2652::DEBUG::2012-12-09 13:03:48,752::task::588::TaskManager.Task::(_updateState) Task=`318ece2e-b699-473f-9a1e-8100defa918b`::moving from state preparing -> state finished
>Thread-2652::DEBUG::2012-12-09 13:03:48,752::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2652::DEBUG::2012-12-09 13:03:48,753::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2652::DEBUG::2012-12-09 13:03:48,753::task::978::TaskManager.Task::(_decref) Task=`318ece2e-b699-473f-9a1e-8100defa918b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:49,180::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:49,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160583 s, 63.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:49,482::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:49,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:49,582::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:49,583::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:49,583::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:49,584::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:03:51,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:51,276::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173018 s, 59.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:03:52,708::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:03:52,719::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000468812 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:53,283::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:53,324::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167093 s, 61.3 MB/s\n'; <rc> = 0
>Thread-2657::DEBUG::2012-12-09 13:03:55,208::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2657::DEBUG::2012-12-09 13:03:55,209::task::588::TaskManager.Task::(_updateState) Task=`82ce30ae-1d53-4d41-8dd3-732c88e5dbcf`::moving from state init -> state preparing
>Thread-2657::INFO::2012-12-09 13:03:55,209::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2657::INFO::2012-12-09 13:03:55,210::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2657::DEBUG::2012-12-09 13:03:55,210::task::1172::TaskManager.Task::(prepare) Task=`82ce30ae-1d53-4d41-8dd3-732c88e5dbcf`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2657::DEBUG::2012-12-09 13:03:55,210::task::588::TaskManager.Task::(_updateState) Task=`82ce30ae-1d53-4d41-8dd3-732c88e5dbcf`::moving from state preparing -> state finished
>Thread-2657::DEBUG::2012-12-09 13:03:55,210::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2657::DEBUG::2012-12-09 13:03:55,210::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2657::DEBUG::2012-12-09 13:03:55,211::task::978::TaskManager.Task::(_decref) Task=`82ce30ae-1d53-4d41-8dd3-732c88e5dbcf`::ref 0 aborting False
>Thread-2658::DEBUG::2012-12-09 13:03:55,225::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2658::DEBUG::2012-12-09 13:03:55,225::task::588::TaskManager.Task::(_updateState) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::moving from state init -> state preparing
>Thread-2658::INFO::2012-12-09 13:03:55,225::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2658::DEBUG::2012-12-09 13:03:55,226::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4946cefa-541e-4a84-b115-29478f52964b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2658::DEBUG::2012-12-09 13:03:55,226::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2658::DEBUG::2012-12-09 13:03:55,226::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2658::DEBUG::2012-12-09 13:03:55,226::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4946cefa-541e-4a84-b115-29478f52964b`::Granted request
>Thread-2658::DEBUG::2012-12-09 13:03:55,227::task::817::TaskManager.Task::(resourceAcquired) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2658::DEBUG::2012-12-09 13:03:55,227::task::978::TaskManager.Task::(_decref) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::ref 1 aborting False
>Thread-2658::DEBUG::2012-12-09 13:03:55,228::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2658::DEBUG::2012-12-09 13:03:55,228::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2658::DEBUG::2012-12-09 13:03:55,327::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2658::WARNING::2012-12-09 13:03:55,328::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2658::DEBUG::2012-12-09 13:03:55,328::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2658::WARNING::2012-12-09 13:03:55,329::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2658::INFO::2012-12-09 13:03:55,329::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2658::DEBUG::2012-12-09 13:03:55,329::task::1172::TaskManager.Task::(prepare) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2658::DEBUG::2012-12-09 13:03:55,329::task::588::TaskManager.Task::(_updateState) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::moving from state preparing -> state finished
>Thread-2658::DEBUG::2012-12-09 13:03:55,330::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2658::DEBUG::2012-12-09 13:03:55,330::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2658::DEBUG::2012-12-09 13:03:55,330::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Dummy-321::DEBUG::2012-12-09 13:03:55,331::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-2658::DEBUG::2012-12-09 13:03:55,331::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2658::DEBUG::2012-12-09 13:03:55,336::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2658::DEBUG::2012-12-09 13:03:55,337::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2658::DEBUG::2012-12-09 13:03:55,337::task::978::TaskManager.Task::(_decref) Task=`43d99739-0cd8-4b13-8376-426d4807f61d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:55,399::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278237 s, 36.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:03:57,409::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:57,448::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163669 s, 62.6 MB/s\n'; <rc> = 0
>Thread-2660::DEBUG::2012-12-09 13:03:58,921::task::588::TaskManager.Task::(_updateState) Task=`c10fb6a9-7352-4d68-b1c0-b98ce3808194`::moving from state init -> state preparing
>Thread-2660::INFO::2012-12-09 13:03:58,921::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2660::INFO::2012-12-09 13:03:58,921::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117881298065', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2660::DEBUG::2012-12-09 13:03:58,921::task::1172::TaskManager.Task::(prepare) Task=`c10fb6a9-7352-4d68-b1c0-b98ce3808194`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117881298065', 'lastCheck': '6.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-2660::DEBUG::2012-12-09 13:03:58,922::task::588::TaskManager.Task::(_updateState) Task=`c10fb6a9-7352-4d68-b1c0-b98ce3808194`::moving from state preparing -> state finished
>Thread-2660::DEBUG::2012-12-09 13:03:58,922::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2660::DEBUG::2012-12-09 13:03:58,922::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2660::DEBUG::2012-12-09 13:03:58,922::task::978::TaskManager.Task::(_decref) Task=`c10fb6a9-7352-4d68-b1c0-b98ce3808194`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:03:59,455::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:03:59,496::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162817 s, 62.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:03:59,585::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:03:59,585::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:03:59,688::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:03:59,690::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:03:59,690::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:03:59,691::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:01,503::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:01,551::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249104 s, 41.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:02,722::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:02,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000586973 s, 7.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:03,557::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:03,598::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016581 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:05,605::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:05,646::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192917 s, 53.1 MB/s\n'; <rc> = 0
>Thread-2665::DEBUG::2012-12-09 13:04:05,667::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2665::DEBUG::2012-12-09 13:04:05,668::task::588::TaskManager.Task::(_updateState) Task=`7b93635b-a6a5-492d-ad0c-932a1f7f2b3d`::moving from state init -> state preparing
>Thread-2665::INFO::2012-12-09 13:04:05,668::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2665::INFO::2012-12-09 13:04:05,669::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2665::DEBUG::2012-12-09 13:04:05,669::task::1172::TaskManager.Task::(prepare) Task=`7b93635b-a6a5-492d-ad0c-932a1f7f2b3d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2665::DEBUG::2012-12-09 13:04:05,669::task::588::TaskManager.Task::(_updateState) Task=`7b93635b-a6a5-492d-ad0c-932a1f7f2b3d`::moving from state preparing -> state finished
>Thread-2665::DEBUG::2012-12-09 13:04:05,669::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2665::DEBUG::2012-12-09 13:04:05,670::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2665::DEBUG::2012-12-09 13:04:05,670::task::978::TaskManager.Task::(_decref) Task=`7b93635b-a6a5-492d-ad0c-932a1f7f2b3d`::ref 0 aborting False
>Thread-2666::DEBUG::2012-12-09 13:04:05,683::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2666::DEBUG::2012-12-09 13:04:05,684::task::588::TaskManager.Task::(_updateState) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::moving from state init -> state preparing
>Thread-2666::INFO::2012-12-09 13:04:05,684::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2666::DEBUG::2012-12-09 13:04:05,684::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0143a854-1059-4b1a-bcb8-0c246d8e7779`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2666::DEBUG::2012-12-09 13:04:05,685::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2666::DEBUG::2012-12-09 13:04:05,685::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2666::DEBUG::2012-12-09 13:04:05,685::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0143a854-1059-4b1a-bcb8-0c246d8e7779`::Granted request
>Thread-2666::DEBUG::2012-12-09 13:04:05,686::task::817::TaskManager.Task::(resourceAcquired) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2666::DEBUG::2012-12-09 13:04:05,686::task::978::TaskManager.Task::(_decref) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::ref 1 aborting False
>Thread-2666::DEBUG::2012-12-09 13:04:05,687::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2666::DEBUG::2012-12-09 13:04:05,687::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2666::DEBUG::2012-12-09 13:04:05,786::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2666::WARNING::2012-12-09 13:04:05,787::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2666::DEBUG::2012-12-09 13:04:05,788::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2666::WARNING::2012-12-09 13:04:05,788::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2666::INFO::2012-12-09 13:04:05,789::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2666::DEBUG::2012-12-09 13:04:05,789::task::1172::TaskManager.Task::(prepare) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2666::DEBUG::2012-12-09 13:04:05,789::task::588::TaskManager.Task::(_updateState) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::moving from state preparing -> state finished
>Thread-2666::DEBUG::2012-12-09 13:04:05,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2666::DEBUG::2012-12-09 13:04:05,790::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2666::DEBUG::2012-12-09 13:04:05,790::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2666::DEBUG::2012-12-09 13:04:05,790::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2666::DEBUG::2012-12-09 13:04:05,790::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2666::DEBUG::2012-12-09 13:04:05,791::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2666::DEBUG::2012-12-09 13:04:05,791::task::978::TaskManager.Task::(_decref) Task=`ca204c4e-42fa-4ff8-837f-aade2d3b7cf1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:07,653::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:07,693::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162908 s, 62.9 MB/s\n'; <rc> = 0
>Thread-2668::DEBUG::2012-12-09 13:04:09,350::task::588::TaskManager.Task::(_updateState) Task=`32338d67-79ce-47a2-aaac-e4ca712486b9`::moving from state init -> state preparing
>Thread-2668::INFO::2012-12-09 13:04:09,350::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2668::INFO::2012-12-09 13:04:09,351::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122900009155', 'lastCheck': '6.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-2668::DEBUG::2012-12-09 13:04:09,351::task::1172::TaskManager.Task::(prepare) Task=`32338d67-79ce-47a2-aaac-e4ca712486b9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122900009155', 'lastCheck': '6.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-2668::DEBUG::2012-12-09 13:04:09,351::task::588::TaskManager.Task::(_updateState) Task=`32338d67-79ce-47a2-aaac-e4ca712486b9`::moving from state preparing -> state finished
>Thread-2668::DEBUG::2012-12-09 13:04:09,351::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2668::DEBUG::2012-12-09 13:04:09,351::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2668::DEBUG::2012-12-09 13:04:09,352::task::978::TaskManager.Task::(_decref) Task=`32338d67-79ce-47a2-aaac-e4ca712486b9`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:04:09,692::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:04:09,693::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:09,700::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:09,775::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0338827 s, 30.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:04:09,845::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:04:09,846::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:04:09,846::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:04:09,847::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:11,784::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:11,831::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194592 s, 52.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:12,737::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 13:04:12,737::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:04:12,738::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:12,749::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000583206 s, 3.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:12,750::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000583206 s, 3.5 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 13:04:12,750::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 13:04:12,750::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 13:04:12,750::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:04:12,751::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,751::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,751::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,751::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,752::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,752::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:12,852::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:12,854::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:04:12,854::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 13:04:12,855::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 13:04:12,855::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 13:04:12,855::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:04:12,855::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:12,952::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:12,952::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:12,962::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000490517 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:13,838::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:13,882::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203222 s, 50.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:15,889::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:15,930::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170496 s, 60.1 MB/s\n'; <rc> = 0
>Thread-2673::DEBUG::2012-12-09 13:04:16,026::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2673::DEBUG::2012-12-09 13:04:16,027::task::588::TaskManager.Task::(_updateState) Task=`65a4fc84-6902-4274-9864-b42b447bdd15`::moving from state init -> state preparing
>Thread-2673::INFO::2012-12-09 13:04:16,027::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2673::INFO::2012-12-09 13:04:16,027::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2673::DEBUG::2012-12-09 13:04:16,028::task::1172::TaskManager.Task::(prepare) Task=`65a4fc84-6902-4274-9864-b42b447bdd15`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2673::DEBUG::2012-12-09 13:04:16,028::task::588::TaskManager.Task::(_updateState) Task=`65a4fc84-6902-4274-9864-b42b447bdd15`::moving from state preparing -> state finished
>Thread-2673::DEBUG::2012-12-09 13:04:16,028::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2673::DEBUG::2012-12-09 13:04:16,028::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2673::DEBUG::2012-12-09 13:04:16,028::task::978::TaskManager.Task::(_decref) Task=`65a4fc84-6902-4274-9864-b42b447bdd15`::ref 0 aborting False
>Thread-2674::DEBUG::2012-12-09 13:04:16,043::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2674::DEBUG::2012-12-09 13:04:16,044::task::588::TaskManager.Task::(_updateState) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::moving from state init -> state preparing
>Thread-2674::INFO::2012-12-09 13:04:16,044::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2674::DEBUG::2012-12-09 13:04:16,044::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`80dbc8cf-76c4-4b9f-9d32-2f19cbbc1409`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2674::DEBUG::2012-12-09 13:04:16,045::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2674::DEBUG::2012-12-09 13:04:16,045::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2674::DEBUG::2012-12-09 13:04:16,045::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`80dbc8cf-76c4-4b9f-9d32-2f19cbbc1409`::Granted request
>Thread-2674::DEBUG::2012-12-09 13:04:16,045::task::817::TaskManager.Task::(resourceAcquired) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2674::DEBUG::2012-12-09 13:04:16,046::task::978::TaskManager.Task::(_decref) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::ref 1 aborting False
>Thread-2674::DEBUG::2012-12-09 13:04:16,046::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2674::DEBUG::2012-12-09 13:04:16,047::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2674::DEBUG::2012-12-09 13:04:16,145::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2674::WARNING::2012-12-09 13:04:16,146::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2674::DEBUG::2012-12-09 13:04:16,146::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2674::WARNING::2012-12-09 13:04:16,147::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2674::INFO::2012-12-09 13:04:16,147::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2674::DEBUG::2012-12-09 13:04:16,147::task::1172::TaskManager.Task::(prepare) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2674::DEBUG::2012-12-09 13:04:16,148::task::588::TaskManager.Task::(_updateState) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::moving from state preparing -> state finished
>Thread-2674::DEBUG::2012-12-09 13:04:16,148::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2674::DEBUG::2012-12-09 13:04:16,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2674::DEBUG::2012-12-09 13:04:16,148::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2674::DEBUG::2012-12-09 13:04:16,149::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2674::DEBUG::2012-12-09 13:04:16,149::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2674::DEBUG::2012-12-09 13:04:16,149::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2674::DEBUG::2012-12-09 13:04:16,149::task::978::TaskManager.Task::(_decref) Task=`17df8b70-a71a-4c8c-a9fa-74e00d3b293e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:17,936::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:17,976::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165314 s, 61.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:04:19,848::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:04:19,848::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2676::DEBUG::2012-12-09 13:04:19,922::task::588::TaskManager.Task::(_updateState) Task=`e304b883-4bc4-4a21-8b40-ada77acf9b86`::moving from state init -> state preparing
>Thread-2676::INFO::2012-12-09 13:04:19,923::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2676::INFO::2012-12-09 13:04:19,924::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103878974915', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-2676::DEBUG::2012-12-09 13:04:19,924::task::1172::TaskManager.Task::(prepare) Task=`e304b883-4bc4-4a21-8b40-ada77acf9b86`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103878974915', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.1', 'code': 358, 'valid': False}}
>Thread-2676::DEBUG::2012-12-09 13:04:19,924::task::588::TaskManager.Task::(_updateState) Task=`e304b883-4bc4-4a21-8b40-ada77acf9b86`::moving from state preparing -> state finished
>Thread-2676::DEBUG::2012-12-09 13:04:19,924::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2676::DEBUG::2012-12-09 13:04:19,925::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2676::DEBUG::2012-12-09 13:04:19,925::task::978::TaskManager.Task::(_decref) Task=`e304b883-4bc4-4a21-8b40-ada77acf9b86`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:04:19,962::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:04:19,963::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:04:19,963::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:04:19,964::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:19,984::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:20,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182837 s, 56.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:22,033::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:22,085::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251449 s, 40.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:22,965::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:22,974::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000431219 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:24,092::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:24,131::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163146 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:26,138::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:26,180::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178087 s, 57.5 MB/s\n'; <rc> = 0
>Thread-2681::DEBUG::2012-12-09 13:04:26,684::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2681::DEBUG::2012-12-09 13:04:26,685::task::588::TaskManager.Task::(_updateState) Task=`5f1fd921-24a4-42dc-a954-ffd2395ec1f7`::moving from state init -> state preparing
>Thread-2681::INFO::2012-12-09 13:04:26,685::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2681::INFO::2012-12-09 13:04:26,686::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2681::DEBUG::2012-12-09 13:04:26,686::task::1172::TaskManager.Task::(prepare) Task=`5f1fd921-24a4-42dc-a954-ffd2395ec1f7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2681::DEBUG::2012-12-09 13:04:26,686::task::588::TaskManager.Task::(_updateState) Task=`5f1fd921-24a4-42dc-a954-ffd2395ec1f7`::moving from state preparing -> state finished
>Thread-2681::DEBUG::2012-12-09 13:04:26,686::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2681::DEBUG::2012-12-09 13:04:26,686::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2681::DEBUG::2012-12-09 13:04:26,687::task::978::TaskManager.Task::(_decref) Task=`5f1fd921-24a4-42dc-a954-ffd2395ec1f7`::ref 0 aborting False
>Thread-2682::DEBUG::2012-12-09 13:04:26,699::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2682::DEBUG::2012-12-09 13:04:26,700::task::588::TaskManager.Task::(_updateState) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::moving from state init -> state preparing
>Thread-2682::INFO::2012-12-09 13:04:26,700::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2682::DEBUG::2012-12-09 13:04:26,701::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f727ca5-dfd9-4b3c-99d6-eb2fb8a630fc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2682::DEBUG::2012-12-09 13:04:26,701::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2682::DEBUG::2012-12-09 13:04:26,701::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2682::DEBUG::2012-12-09 13:04:26,702::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0f727ca5-dfd9-4b3c-99d6-eb2fb8a630fc`::Granted request
>Thread-2682::DEBUG::2012-12-09 13:04:26,702::task::817::TaskManager.Task::(resourceAcquired) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2682::DEBUG::2012-12-09 13:04:26,702::task::978::TaskManager.Task::(_decref) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::ref 1 aborting False
>Thread-2682::DEBUG::2012-12-09 13:04:26,704::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2682::DEBUG::2012-12-09 13:04:26,704::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2682::DEBUG::2012-12-09 13:04:26,808::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2682::WARNING::2012-12-09 13:04:26,809::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2682::DEBUG::2012-12-09 13:04:26,809::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2682::WARNING::2012-12-09 13:04:26,810::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2682::INFO::2012-12-09 13:04:26,810::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2682::DEBUG::2012-12-09 13:04:26,810::task::1172::TaskManager.Task::(prepare) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2682::DEBUG::2012-12-09 13:04:26,811::task::588::TaskManager.Task::(_updateState) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::moving from state preparing -> state finished
>Thread-2682::DEBUG::2012-12-09 13:04:26,811::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2682::DEBUG::2012-12-09 13:04:26,811::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2682::DEBUG::2012-12-09 13:04:26,811::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2682::DEBUG::2012-12-09 13:04:26,812::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2682::DEBUG::2012-12-09 13:04:26,812::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2682::DEBUG::2012-12-09 13:04:26,812::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2682::DEBUG::2012-12-09 13:04:26,812::task::978::TaskManager.Task::(_decref) Task=`35743617-04c0-4c0c-9663-710733b3ff97`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:28,186::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:28,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161934 s, 63.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:04:29,964::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:04:29,965::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:04:30,066::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:04:30,067::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:04:30,067::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:04:30,068::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:30,234::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:30,279::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163638 s, 62.6 MB/s\n'; <rc> = 0
>Thread-2684::DEBUG::2012-12-09 13:04:30,283::task::588::TaskManager.Task::(_updateState) Task=`0624d94c-c654-4a5e-86fb-63a861982322`::moving from state init -> state preparing
>Thread-2684::INFO::2012-12-09 13:04:30,285::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2684::INFO::2012-12-09 13:04:30,289::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091381072998', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-2684::DEBUG::2012-12-09 13:04:30,291::task::1172::TaskManager.Task::(prepare) Task=`0624d94c-c654-4a5e-86fb-63a861982322`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091381072998', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-2684::DEBUG::2012-12-09 13:04:30,292::task::588::TaskManager.Task::(_updateState) Task=`0624d94c-c654-4a5e-86fb-63a861982322`::moving from state preparing -> state finished
>Thread-2684::DEBUG::2012-12-09 13:04:30,292::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2684::DEBUG::2012-12-09 13:04:30,292::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2684::DEBUG::2012-12-09 13:04:30,292::task::978::TaskManager.Task::(_decref) Task=`0624d94c-c654-4a5e-86fb-63a861982322`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:32,294::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:32,333::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155624 s, 65.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:32,977::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:32,988::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000521948 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:34,340::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:34,384::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211018 s, 48.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:36,391::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:36,433::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018628 s, 55.0 MB/s\n'; <rc> = 0
>Thread-2689::DEBUG::2012-12-09 13:04:37,164::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2689::DEBUG::2012-12-09 13:04:37,165::task::588::TaskManager.Task::(_updateState) Task=`1d225ab0-4677-446f-a7ed-87272753d580`::moving from state init -> state preparing
>Thread-2689::INFO::2012-12-09 13:04:37,165::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2689::INFO::2012-12-09 13:04:37,166::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2689::DEBUG::2012-12-09 13:04:37,166::task::1172::TaskManager.Task::(prepare) Task=`1d225ab0-4677-446f-a7ed-87272753d580`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2689::DEBUG::2012-12-09 13:04:37,167::task::588::TaskManager.Task::(_updateState) Task=`1d225ab0-4677-446f-a7ed-87272753d580`::moving from state preparing -> state finished
>Thread-2689::DEBUG::2012-12-09 13:04:37,167::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2689::DEBUG::2012-12-09 13:04:37,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2689::DEBUG::2012-12-09 13:04:37,167::task::978::TaskManager.Task::(_decref) Task=`1d225ab0-4677-446f-a7ed-87272753d580`::ref 0 aborting False
>Thread-2690::DEBUG::2012-12-09 13:04:37,180::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2690::DEBUG::2012-12-09 13:04:37,181::task::588::TaskManager.Task::(_updateState) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::moving from state init -> state preparing
>Thread-2690::INFO::2012-12-09 13:04:37,181::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2690::DEBUG::2012-12-09 13:04:37,182::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c4c6a2a-7cd4-4e3d-88c7-d9eccb9c9588`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2690::DEBUG::2012-12-09 13:04:37,182::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2690::DEBUG::2012-12-09 13:04:37,182::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2690::DEBUG::2012-12-09 13:04:37,182::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c4c6a2a-7cd4-4e3d-88c7-d9eccb9c9588`::Granted request
>Thread-2690::DEBUG::2012-12-09 13:04:37,183::task::817::TaskManager.Task::(resourceAcquired) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2690::DEBUG::2012-12-09 13:04:37,183::task::978::TaskManager.Task::(_decref) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::ref 1 aborting False
>Thread-2690::DEBUG::2012-12-09 13:04:37,184::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2690::DEBUG::2012-12-09 13:04:37,184::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2690::DEBUG::2012-12-09 13:04:37,283::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2690::WARNING::2012-12-09 13:04:37,284::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2690::DEBUG::2012-12-09 13:04:37,284::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2690::WARNING::2012-12-09 13:04:37,285::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2690::INFO::2012-12-09 13:04:37,285::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2690::DEBUG::2012-12-09 13:04:37,285::task::1172::TaskManager.Task::(prepare) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2690::DEBUG::2012-12-09 13:04:37,286::task::588::TaskManager.Task::(_updateState) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::moving from state preparing -> state finished
>Thread-2690::DEBUG::2012-12-09 13:04:37,286::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2690::DEBUG::2012-12-09 13:04:37,286::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2690::DEBUG::2012-12-09 13:04:37,287::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2690::DEBUG::2012-12-09 13:04:37,287::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2690::DEBUG::2012-12-09 13:04:37,287::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2690::DEBUG::2012-12-09 13:04:37,287::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2690::DEBUG::2012-12-09 13:04:37,288::task::978::TaskManager.Task::(_decref) Task=`c0c2da7e-2819-46c3-bd79-cdbbfd52966f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:38,440::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:38,478::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162721 s, 62.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:04:40,068::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:04:40,069::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:04:40,070::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:04:40,172::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:04:40,173::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:04:40,173::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:04:40,174::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:40,485::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:40,529::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165046 s, 62.0 MB/s\n'; <rc> = 0
>Thread-2692::DEBUG::2012-12-09 13:04:40,537::task::588::TaskManager.Task::(_updateState) Task=`d822c3fc-04c1-4b81-b4dc-16223ed8f473`::moving from state init -> state preparing
>Thread-2692::INFO::2012-12-09 13:04:40,538::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2692::INFO::2012-12-09 13:04:40,538::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0115649700165', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.4', 'code': 358, 'valid': False}}
>Thread-2692::DEBUG::2012-12-09 13:04:40,538::task::1172::TaskManager.Task::(prepare) Task=`d822c3fc-04c1-4b81-b4dc-16223ed8f473`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0115649700165', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.4', 'code': 358, 'valid': False}}
>Thread-2692::DEBUG::2012-12-09 13:04:40,539::task::588::TaskManager.Task::(_updateState) Task=`d822c3fc-04c1-4b81-b4dc-16223ed8f473`::moving from state preparing -> state finished
>Thread-2692::DEBUG::2012-12-09 13:04:40,539::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2692::DEBUG::2012-12-09 13:04:40,539::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2692::DEBUG::2012-12-09 13:04:40,539::task::978::TaskManager.Task::(_decref) Task=`d822c3fc-04c1-4b81-b4dc-16223ed8f473`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:42,537::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:42,576::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164578 s, 62.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:42,991::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:42,999::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000500232 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:44,583::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:44,622::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016316 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:46,629::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:46,670::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163978 s, 62.4 MB/s\n'; <rc> = 0
>Thread-2697::DEBUG::2012-12-09 13:04:47,439::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2697::DEBUG::2012-12-09 13:04:47,439::task::588::TaskManager.Task::(_updateState) Task=`f27c71d1-a104-4938-a0d9-3fcf432a8edd`::moving from state init -> state preparing
>Thread-2697::INFO::2012-12-09 13:04:47,440::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2697::INFO::2012-12-09 13:04:47,440::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2697::DEBUG::2012-12-09 13:04:47,440::task::1172::TaskManager.Task::(prepare) Task=`f27c71d1-a104-4938-a0d9-3fcf432a8edd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2697::DEBUG::2012-12-09 13:04:47,440::task::588::TaskManager.Task::(_updateState) Task=`f27c71d1-a104-4938-a0d9-3fcf432a8edd`::moving from state preparing -> state finished
>Thread-2697::DEBUG::2012-12-09 13:04:47,440::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2697::DEBUG::2012-12-09 13:04:47,441::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2697::DEBUG::2012-12-09 13:04:47,441::task::978::TaskManager.Task::(_decref) Task=`f27c71d1-a104-4938-a0d9-3fcf432a8edd`::ref 0 aborting False
>Thread-2698::DEBUG::2012-12-09 13:04:47,454::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2698::DEBUG::2012-12-09 13:04:47,454::task::588::TaskManager.Task::(_updateState) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::moving from state init -> state preparing
>Thread-2698::INFO::2012-12-09 13:04:47,455::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2698::DEBUG::2012-12-09 13:04:47,455::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c958433-7bd0-40ad-9771-943807f34214`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2698::DEBUG::2012-12-09 13:04:47,455::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2698::DEBUG::2012-12-09 13:04:47,456::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2698::DEBUG::2012-12-09 13:04:47,456::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4c958433-7bd0-40ad-9771-943807f34214`::Granted request
>Thread-2698::DEBUG::2012-12-09 13:04:47,457::task::817::TaskManager.Task::(resourceAcquired) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2698::DEBUG::2012-12-09 13:04:47,457::task::978::TaskManager.Task::(_decref) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::ref 1 aborting False
>Thread-2698::DEBUG::2012-12-09 13:04:47,458::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2698::DEBUG::2012-12-09 13:04:47,459::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2698::DEBUG::2012-12-09 13:04:47,555::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2698::WARNING::2012-12-09 13:04:47,556::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2698::DEBUG::2012-12-09 13:04:47,556::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2698::WARNING::2012-12-09 13:04:47,557::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2698::INFO::2012-12-09 13:04:47,557::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2698::DEBUG::2012-12-09 13:04:47,557::task::1172::TaskManager.Task::(prepare) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2698::DEBUG::2012-12-09 13:04:47,557::task::588::TaskManager.Task::(_updateState) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::moving from state preparing -> state finished
>Thread-2698::DEBUG::2012-12-09 13:04:47,558::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2698::DEBUG::2012-12-09 13:04:47,558::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2698::DEBUG::2012-12-09 13:04:47,558::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2698::DEBUG::2012-12-09 13:04:47,558::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2698::DEBUG::2012-12-09 13:04:47,559::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2698::DEBUG::2012-12-09 13:04:47,559::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2698::DEBUG::2012-12-09 13:04:47,559::task::978::TaskManager.Task::(_decref) Task=`eb52a2f0-9ca9-4a97-929c-652c46aef22d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:48,677::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:48,723::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172183 s, 59.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:04:50,175::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:04:50,175::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:04:50,273::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:04:50,275::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:04:50,275::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:04:50,275::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:04:50,730::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-2700::DEBUG::2012-12-09 13:04:50,752::task::588::TaskManager.Task::(_updateState) Task=`b16339c6-0738-45cf-81c7-d6fca8313f6c`::moving from state init -> state preparing
>Thread-2700::INFO::2012-12-09 13:04:50,753::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2700::INFO::2012-12-09 13:04:50,754::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00856995582581', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-2700::DEBUG::2012-12-09 13:04:50,754::task::1172::TaskManager.Task::(prepare) Task=`b16339c6-0738-45cf-81c7-d6fca8313f6c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00856995582581', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-2700::DEBUG::2012-12-09 13:04:50,755::task::588::TaskManager.Task::(_updateState) Task=`b16339c6-0738-45cf-81c7-d6fca8313f6c`::moving from state preparing -> state finished
>Thread-2700::DEBUG::2012-12-09 13:04:50,756::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2700::DEBUG::2012-12-09 13:04:50,756::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2700::DEBUG::2012-12-09 13:04:50,757::task::978::TaskManager.Task::(_decref) Task=`b16339c6-0738-45cf-81c7-d6fca8313f6c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:50,795::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0325306 s, 31.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:52,802::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:52,846::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174834 s, 58.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:04:53,002::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:04:53,013::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000501764 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:54,853::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:54,909::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168996 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:04:56,919::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:56,960::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172654 s, 59.3 MB/s\n'; <rc> = 0
>Thread-2705::DEBUG::2012-12-09 13:04:57,931::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2705::DEBUG::2012-12-09 13:04:57,931::task::588::TaskManager.Task::(_updateState) Task=`4d99b1a2-fc5e-4127-aa49-8ca8bde0b2a1`::moving from state init -> state preparing
>Thread-2705::INFO::2012-12-09 13:04:57,932::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2705::INFO::2012-12-09 13:04:57,932::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2705::DEBUG::2012-12-09 13:04:57,932::task::1172::TaskManager.Task::(prepare) Task=`4d99b1a2-fc5e-4127-aa49-8ca8bde0b2a1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2705::DEBUG::2012-12-09 13:04:57,932::task::588::TaskManager.Task::(_updateState) Task=`4d99b1a2-fc5e-4127-aa49-8ca8bde0b2a1`::moving from state preparing -> state finished
>Thread-2705::DEBUG::2012-12-09 13:04:57,932::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2705::DEBUG::2012-12-09 13:04:57,933::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2705::DEBUG::2012-12-09 13:04:57,933::task::978::TaskManager.Task::(_decref) Task=`4d99b1a2-fc5e-4127-aa49-8ca8bde0b2a1`::ref 0 aborting False
>Thread-2706::DEBUG::2012-12-09 13:04:57,949::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2706::DEBUG::2012-12-09 13:04:57,949::task::588::TaskManager.Task::(_updateState) Task=`2009cb19-b932-4e6d-9159-20655603307f`::moving from state init -> state preparing
>Thread-2706::INFO::2012-12-09 13:04:57,950::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2706::DEBUG::2012-12-09 13:04:57,950::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c44644fb-8eba-42da-8e20-cfe70f294d85`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2706::DEBUG::2012-12-09 13:04:57,950::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2706::DEBUG::2012-12-09 13:04:57,950::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2706::DEBUG::2012-12-09 13:04:57,951::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c44644fb-8eba-42da-8e20-cfe70f294d85`::Granted request
>Thread-2706::DEBUG::2012-12-09 13:04:57,952::task::817::TaskManager.Task::(resourceAcquired) Task=`2009cb19-b932-4e6d-9159-20655603307f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2706::DEBUG::2012-12-09 13:04:57,952::task::978::TaskManager.Task::(_decref) Task=`2009cb19-b932-4e6d-9159-20655603307f`::ref 1 aborting False
>Thread-2706::DEBUG::2012-12-09 13:04:57,953::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2706::DEBUG::2012-12-09 13:04:57,953::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2706::DEBUG::2012-12-09 13:04:58,052::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2706::WARNING::2012-12-09 13:04:58,053::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2706::DEBUG::2012-12-09 13:04:58,054::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2706::WARNING::2012-12-09 13:04:58,054::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2706::INFO::2012-12-09 13:04:58,054::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2706::DEBUG::2012-12-09 13:04:58,055::task::1172::TaskManager.Task::(prepare) Task=`2009cb19-b932-4e6d-9159-20655603307f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2706::DEBUG::2012-12-09 13:04:58,055::task::588::TaskManager.Task::(_updateState) Task=`2009cb19-b932-4e6d-9159-20655603307f`::moving from state preparing -> state finished
>Thread-2706::DEBUG::2012-12-09 13:04:58,055::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2706::DEBUG::2012-12-09 13:04:58,055::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2706::DEBUG::2012-12-09 13:04:58,055::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2706::DEBUG::2012-12-09 13:04:58,056::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2706::DEBUG::2012-12-09 13:04:58,056::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2706::DEBUG::2012-12-09 13:04:58,056::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2706::DEBUG::2012-12-09 13:04:58,056::task::978::TaskManager.Task::(_decref) Task=`2009cb19-b932-4e6d-9159-20655603307f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:04:58,967::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:04:59,012::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207918 s, 49.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:00,276::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:00,277::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:00,375::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:00,376::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:00,376::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:00,377::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:01,019::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:01,058::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164203 s, 62.4 MB/s\n'; <rc> = 0
>Thread-2708::DEBUG::2012-12-09 13:05:01,170::task::588::TaskManager.Task::(_updateState) Task=`123d96fd-71aa-44d2-8cd0-bfe5c366f99b`::moving from state init -> state preparing
>Thread-2708::INFO::2012-12-09 13:05:01,170::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2708::INFO::2012-12-09 13:05:01,171::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116322040558', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-2708::DEBUG::2012-12-09 13:05:01,171::task::1172::TaskManager.Task::(prepare) Task=`123d96fd-71aa-44d2-8cd0-bfe5c366f99b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0116322040558', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-2708::DEBUG::2012-12-09 13:05:01,171::task::588::TaskManager.Task::(_updateState) Task=`123d96fd-71aa-44d2-8cd0-bfe5c366f99b`::moving from state preparing -> state finished
>Thread-2708::DEBUG::2012-12-09 13:05:01,171::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2708::DEBUG::2012-12-09 13:05:01,172::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2708::DEBUG::2012-12-09 13:05:01,172::task::978::TaskManager.Task::(_decref) Task=`123d96fd-71aa-44d2-8cd0-bfe5c366f99b`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:05:03,016::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:03,028::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000413684 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:03,065::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:03,105::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168135 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:05,112::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:05,153::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171688 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:07,159::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:07,204::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200223 s, 51.1 MB/s\n'; <rc> = 0
>Thread-2713::DEBUG::2012-12-09 13:05:08,295::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2713::DEBUG::2012-12-09 13:05:08,296::task::588::TaskManager.Task::(_updateState) Task=`7eb40b4a-3fe0-44d7-a0e1-d4e34136cd7f`::moving from state init -> state preparing
>Thread-2713::INFO::2012-12-09 13:05:08,296::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2713::INFO::2012-12-09 13:05:08,296::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2713::DEBUG::2012-12-09 13:05:08,297::task::1172::TaskManager.Task::(prepare) Task=`7eb40b4a-3fe0-44d7-a0e1-d4e34136cd7f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2713::DEBUG::2012-12-09 13:05:08,297::task::588::TaskManager.Task::(_updateState) Task=`7eb40b4a-3fe0-44d7-a0e1-d4e34136cd7f`::moving from state preparing -> state finished
>Thread-2713::DEBUG::2012-12-09 13:05:08,297::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2713::DEBUG::2012-12-09 13:05:08,297::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2713::DEBUG::2012-12-09 13:05:08,297::task::978::TaskManager.Task::(_decref) Task=`7eb40b4a-3fe0-44d7-a0e1-d4e34136cd7f`::ref 0 aborting False
>Thread-2714::DEBUG::2012-12-09 13:05:08,310::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2714::DEBUG::2012-12-09 13:05:08,310::task::588::TaskManager.Task::(_updateState) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::moving from state init -> state preparing
>Thread-2714::INFO::2012-12-09 13:05:08,311::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2714::DEBUG::2012-12-09 13:05:08,311::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`647957cd-c96c-47b6-9a3e-da1d17874b22`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2714::DEBUG::2012-12-09 13:05:08,311::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2714::DEBUG::2012-12-09 13:05:08,312::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2714::DEBUG::2012-12-09 13:05:08,312::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`647957cd-c96c-47b6-9a3e-da1d17874b22`::Granted request
>Thread-2714::DEBUG::2012-12-09 13:05:08,312::task::817::TaskManager.Task::(resourceAcquired) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2714::DEBUG::2012-12-09 13:05:08,312::task::978::TaskManager.Task::(_decref) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::ref 1 aborting False
>Thread-2714::DEBUG::2012-12-09 13:05:08,313::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2714::DEBUG::2012-12-09 13:05:08,313::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2714::DEBUG::2012-12-09 13:05:08,413::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2714::WARNING::2012-12-09 13:05:08,414::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2714::DEBUG::2012-12-09 13:05:08,415::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2714::WARNING::2012-12-09 13:05:08,415::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2714::INFO::2012-12-09 13:05:08,415::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2714::DEBUG::2012-12-09 13:05:08,416::task::1172::TaskManager.Task::(prepare) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2714::DEBUG::2012-12-09 13:05:08,416::task::588::TaskManager.Task::(_updateState) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::moving from state preparing -> state finished
>Thread-2714::DEBUG::2012-12-09 13:05:08,416::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2714::DEBUG::2012-12-09 13:05:08,416::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2714::DEBUG::2012-12-09 13:05:08,417::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2714::DEBUG::2012-12-09 13:05:08,417::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2714::DEBUG::2012-12-09 13:05:08,417::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2714::DEBUG::2012-12-09 13:05:08,417::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2714::DEBUG::2012-12-09 13:05:08,417::task::978::TaskManager.Task::(_decref) Task=`9019bb40-34f6-417c-a3eb-a9bfdd4db65f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:09,210::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:09,252::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177202 s, 57.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:10,378::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:10,378::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:10,478::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:10,480::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:10,480::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:10,480::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:11,258::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:11,300::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173148 s, 59.1 MB/s\n'; <rc> = 0
>Thread-2716::DEBUG::2012-12-09 13:05:11,556::task::588::TaskManager.Task::(_updateState) Task=`2f5c6098-da3d-4c6c-b59f-224342a4955c`::moving from state init -> state preparing
>Thread-2716::INFO::2012-12-09 13:05:11,556::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2716::INFO::2012-12-09 13:05:11,556::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123319625854', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2716::DEBUG::2012-12-09 13:05:11,557::task::1172::TaskManager.Task::(prepare) Task=`2f5c6098-da3d-4c6c-b59f-224342a4955c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123319625854', 'lastCheck': '8.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2716::DEBUG::2012-12-09 13:05:11,557::task::588::TaskManager.Task::(_updateState) Task=`2f5c6098-da3d-4c6c-b59f-224342a4955c`::moving from state preparing -> state finished
>Thread-2716::DEBUG::2012-12-09 13:05:11,557::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2716::DEBUG::2012-12-09 13:05:11,557::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2716::DEBUG::2012-12-09 13:05:11,557::task::978::TaskManager.Task::(_decref) Task=`2f5c6098-da3d-4c6c-b59f-224342a4955c`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:05:13,031::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:13,042::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000470263 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:13,306::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:13,348::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165651 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:15,354::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:15,395::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166856 s, 61.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:17,402::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:17,441::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166264 s, 61.6 MB/s\n'; <rc> = 0
>Thread-2721::DEBUG::2012-12-09 13:05:18,918::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2721::DEBUG::2012-12-09 13:05:18,919::task::588::TaskManager.Task::(_updateState) Task=`cbaaa2e4-21c6-4121-ac0e-01ce2255a244`::moving from state init -> state preparing
>Thread-2721::INFO::2012-12-09 13:05:18,920::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2721::INFO::2012-12-09 13:05:18,920::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2721::DEBUG::2012-12-09 13:05:18,920::task::1172::TaskManager.Task::(prepare) Task=`cbaaa2e4-21c6-4121-ac0e-01ce2255a244`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2721::DEBUG::2012-12-09 13:05:18,920::task::588::TaskManager.Task::(_updateState) Task=`cbaaa2e4-21c6-4121-ac0e-01ce2255a244`::moving from state preparing -> state finished
>Thread-2721::DEBUG::2012-12-09 13:05:18,920::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2721::DEBUG::2012-12-09 13:05:18,921::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2721::DEBUG::2012-12-09 13:05:18,921::task::978::TaskManager.Task::(_decref) Task=`cbaaa2e4-21c6-4121-ac0e-01ce2255a244`::ref 0 aborting False
>Thread-2722::DEBUG::2012-12-09 13:05:18,933::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2722::DEBUG::2012-12-09 13:05:18,934::task::588::TaskManager.Task::(_updateState) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::moving from state init -> state preparing
>Thread-2722::INFO::2012-12-09 13:05:18,934::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2722::DEBUG::2012-12-09 13:05:18,935::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f1f45f0c-5321-4d2d-8115-1ee46844d1ab`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2722::DEBUG::2012-12-09 13:05:18,935::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2722::DEBUG::2012-12-09 13:05:18,935::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2722::DEBUG::2012-12-09 13:05:18,935::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f1f45f0c-5321-4d2d-8115-1ee46844d1ab`::Granted request
>Thread-2722::DEBUG::2012-12-09 13:05:18,936::task::817::TaskManager.Task::(resourceAcquired) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2722::DEBUG::2012-12-09 13:05:18,936::task::978::TaskManager.Task::(_decref) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::ref 1 aborting False
>Thread-2722::DEBUG::2012-12-09 13:05:18,937::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2722::DEBUG::2012-12-09 13:05:18,937::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2722::DEBUG::2012-12-09 13:05:19,041::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2722::WARNING::2012-12-09 13:05:19,043::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2722::DEBUG::2012-12-09 13:05:19,043::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2722::WARNING::2012-12-09 13:05:19,043::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2722::INFO::2012-12-09 13:05:19,044::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2722::DEBUG::2012-12-09 13:05:19,044::task::1172::TaskManager.Task::(prepare) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2722::DEBUG::2012-12-09 13:05:19,044::task::588::TaskManager.Task::(_updateState) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::moving from state preparing -> state finished
>Thread-2722::DEBUG::2012-12-09 13:05:19,045::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2722::DEBUG::2012-12-09 13:05:19,045::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2722::DEBUG::2012-12-09 13:05:19,045::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2722::DEBUG::2012-12-09 13:05:19,045::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2722::DEBUG::2012-12-09 13:05:19,045::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2722::DEBUG::2012-12-09 13:05:19,046::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2722::DEBUG::2012-12-09 13:05:19,046::task::978::TaskManager.Task::(_decref) Task=`92eb0818-60a8-4079-b2b7-86af8efec0ab`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:19,448::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:19,494::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213966 s, 47.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:20,481::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:20,482::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:20,581::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:20,582::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:20,583::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:20,583::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:21,501::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:21,543::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187708 s, 54.6 MB/s\n'; <rc> = 0
>Thread-2724::DEBUG::2012-12-09 13:05:22,043::task::588::TaskManager.Task::(_updateState) Task=`ebaf2cf9-6813-4933-9655-a6d01a4ff671`::moving from state init -> state preparing
>Thread-2724::INFO::2012-12-09 13:05:22,044::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2724::INFO::2012-12-09 13:05:22,044::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118269920349', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-2724::DEBUG::2012-12-09 13:05:22,044::task::1172::TaskManager.Task::(prepare) Task=`ebaf2cf9-6813-4933-9655-a6d01a4ff671`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118269920349', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-2724::DEBUG::2012-12-09 13:05:22,045::task::588::TaskManager.Task::(_updateState) Task=`ebaf2cf9-6813-4933-9655-a6d01a4ff671`::moving from state preparing -> state finished
>Thread-2724::DEBUG::2012-12-09 13:05:22,045::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2724::DEBUG::2012-12-09 13:05:22,045::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2724::DEBUG::2012-12-09 13:05:22,045::task::978::TaskManager.Task::(_decref) Task=`ebaf2cf9-6813-4933-9655-a6d01a4ff671`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:05:23,045::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:23,053::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000420621 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:23,550::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:23,589::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163752 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:25,596::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:25,636::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171086 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:27,643::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:27,683::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172201 s, 59.5 MB/s\n'; <rc> = 0
>Thread-2729::DEBUG::2012-12-09 13:05:29,227::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2729::DEBUG::2012-12-09 13:05:29,227::task::588::TaskManager.Task::(_updateState) Task=`58bfb5ca-81a9-44e5-bb00-74ea01c5508e`::moving from state init -> state preparing
>Thread-2729::INFO::2012-12-09 13:05:29,227::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2729::INFO::2012-12-09 13:05:29,228::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2729::DEBUG::2012-12-09 13:05:29,228::task::1172::TaskManager.Task::(prepare) Task=`58bfb5ca-81a9-44e5-bb00-74ea01c5508e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2729::DEBUG::2012-12-09 13:05:29,228::task::588::TaskManager.Task::(_updateState) Task=`58bfb5ca-81a9-44e5-bb00-74ea01c5508e`::moving from state preparing -> state finished
>Thread-2729::DEBUG::2012-12-09 13:05:29,228::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2729::DEBUG::2012-12-09 13:05:29,229::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2729::DEBUG::2012-12-09 13:05:29,229::task::978::TaskManager.Task::(_decref) Task=`58bfb5ca-81a9-44e5-bb00-74ea01c5508e`::ref 0 aborting False
>Thread-2730::DEBUG::2012-12-09 13:05:29,241::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2730::DEBUG::2012-12-09 13:05:29,241::task::588::TaskManager.Task::(_updateState) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::moving from state init -> state preparing
>Thread-2730::INFO::2012-12-09 13:05:29,242::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2730::DEBUG::2012-12-09 13:05:29,242::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`12800028-7ea5-46a5-91f1-3f883837f331`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2730::DEBUG::2012-12-09 13:05:29,242::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2730::DEBUG::2012-12-09 13:05:29,242::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2730::DEBUG::2012-12-09 13:05:29,243::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`12800028-7ea5-46a5-91f1-3f883837f331`::Granted request
>Thread-2730::DEBUG::2012-12-09 13:05:29,243::task::817::TaskManager.Task::(resourceAcquired) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2730::DEBUG::2012-12-09 13:05:29,243::task::978::TaskManager.Task::(_decref) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::ref 1 aborting False
>Thread-2730::DEBUG::2012-12-09 13:05:29,244::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2730::DEBUG::2012-12-09 13:05:29,244::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2730::DEBUG::2012-12-09 13:05:29,344::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2730::WARNING::2012-12-09 13:05:29,345::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2730::DEBUG::2012-12-09 13:05:29,346::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2730::WARNING::2012-12-09 13:05:29,346::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2730::INFO::2012-12-09 13:05:29,346::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2730::DEBUG::2012-12-09 13:05:29,347::task::1172::TaskManager.Task::(prepare) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2730::DEBUG::2012-12-09 13:05:29,347::task::588::TaskManager.Task::(_updateState) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::moving from state preparing -> state finished
>Thread-2730::DEBUG::2012-12-09 13:05:29,347::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2730::DEBUG::2012-12-09 13:05:29,347::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2730::DEBUG::2012-12-09 13:05:29,348::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2730::DEBUG::2012-12-09 13:05:29,348::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2730::DEBUG::2012-12-09 13:05:29,348::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2730::DEBUG::2012-12-09 13:05:29,348::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2730::DEBUG::2012-12-09 13:05:29,348::task::978::TaskManager.Task::(_decref) Task=`ee8c7415-1343-4133-80cc-c26c3f6c0608`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:29,690::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:29,740::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250959 s, 40.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:30,584::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:30,585::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:30,688::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:30,689::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:30,690::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:30,690::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:31,747::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:31,787::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016653 s, 61.5 MB/s\n'; <rc> = 0
>Thread-2732::DEBUG::2012-12-09 13:05:32,698::task::588::TaskManager.Task::(_updateState) Task=`348d96f8-74ba-47ad-a89b-82aff8fe13f1`::moving from state init -> state preparing
>Thread-2732::INFO::2012-12-09 13:05:32,698::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2732::INFO::2012-12-09 13:05:32,699::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00818610191345', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-2732::DEBUG::2012-12-09 13:05:32,699::task::1172::TaskManager.Task::(prepare) Task=`348d96f8-74ba-47ad-a89b-82aff8fe13f1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00818610191345', 'lastCheck': '9.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-2732::DEBUG::2012-12-09 13:05:32,699::task::588::TaskManager.Task::(_updateState) Task=`348d96f8-74ba-47ad-a89b-82aff8fe13f1`::moving from state preparing -> state finished
>Thread-2732::DEBUG::2012-12-09 13:05:32,699::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2732::DEBUG::2012-12-09 13:05:32,700::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2732::DEBUG::2012-12-09 13:05:32,700::task::978::TaskManager.Task::(_decref) Task=`348d96f8-74ba-47ad-a89b-82aff8fe13f1`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:05:33,056::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:33,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000520161 s, 7.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:33,794::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:33,841::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235977 s, 43.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:35,848::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:35,898::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209863 s, 48.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:37,904::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:37,949::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019491 s, 52.5 MB/s\n'; <rc> = 0
>Thread-2736::DEBUG::2012-12-09 13:05:39,651::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2736::DEBUG::2012-12-09 13:05:39,651::task::588::TaskManager.Task::(_updateState) Task=`a283f2e3-f9b5-4943-b700-8be3a614c37e`::moving from state init -> state preparing
>Thread-2736::INFO::2012-12-09 13:05:39,651::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2736::INFO::2012-12-09 13:05:39,652::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2736::DEBUG::2012-12-09 13:05:39,652::task::1172::TaskManager.Task::(prepare) Task=`a283f2e3-f9b5-4943-b700-8be3a614c37e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2736::DEBUG::2012-12-09 13:05:39,652::task::588::TaskManager.Task::(_updateState) Task=`a283f2e3-f9b5-4943-b700-8be3a614c37e`::moving from state preparing -> state finished
>Thread-2736::DEBUG::2012-12-09 13:05:39,652::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2736::DEBUG::2012-12-09 13:05:39,652::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2736::DEBUG::2012-12-09 13:05:39,653::task::978::TaskManager.Task::(_decref) Task=`a283f2e3-f9b5-4943-b700-8be3a614c37e`::ref 0 aborting False
>Thread-2737::DEBUG::2012-12-09 13:05:39,667::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2737::DEBUG::2012-12-09 13:05:39,667::task::588::TaskManager.Task::(_updateState) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::moving from state init -> state preparing
>Thread-2737::INFO::2012-12-09 13:05:39,667::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2737::DEBUG::2012-12-09 13:05:39,668::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b5c7e2a4-a2f2-4cd9-84e7-d2fbae9376c5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2737::DEBUG::2012-12-09 13:05:39,668::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2737::DEBUG::2012-12-09 13:05:39,668::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2737::DEBUG::2012-12-09 13:05:39,668::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b5c7e2a4-a2f2-4cd9-84e7-d2fbae9376c5`::Granted request
>Thread-2737::DEBUG::2012-12-09 13:05:39,669::task::817::TaskManager.Task::(resourceAcquired) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2737::DEBUG::2012-12-09 13:05:39,669::task::978::TaskManager.Task::(_decref) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::ref 1 aborting False
>Thread-2737::DEBUG::2012-12-09 13:05:39,670::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2737::DEBUG::2012-12-09 13:05:39,670::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2737::DEBUG::2012-12-09 13:05:39,768::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2737::WARNING::2012-12-09 13:05:39,769::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2737::DEBUG::2012-12-09 13:05:39,769::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2737::WARNING::2012-12-09 13:05:39,770::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2737::INFO::2012-12-09 13:05:39,770::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2737::DEBUG::2012-12-09 13:05:39,770::task::1172::TaskManager.Task::(prepare) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2737::DEBUG::2012-12-09 13:05:39,770::task::588::TaskManager.Task::(_updateState) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::moving from state preparing -> state finished
>Thread-2737::DEBUG::2012-12-09 13:05:39,771::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2737::DEBUG::2012-12-09 13:05:39,771::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2737::DEBUG::2012-12-09 13:05:39,771::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2737::DEBUG::2012-12-09 13:05:39,772::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2737::DEBUG::2012-12-09 13:05:39,772::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2737::DEBUG::2012-12-09 13:05:39,772::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2737::DEBUG::2012-12-09 13:05:39,772::task::978::TaskManager.Task::(_decref) Task=`30dd5788-89c0-4c07-a00e-0326045576e4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:39,957::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:39,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164445 s, 62.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:40,691::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:40,691::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:40,791::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:40,792::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:40,792::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:40,793::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:42,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:42,046::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189791 s, 54.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:05:43,070::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:43,086::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000933603 s, 4.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:44,052::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:44,092::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159375 s, 64.3 MB/s\n'; <rc> = 0
>Thread-2740::DEBUG::2012-12-09 13:05:44,943::task::588::TaskManager.Task::(_updateState) Task=`53e412f0-36e3-475f-8fdf-440b8dc33e45`::moving from state init -> state preparing
>Thread-2740::INFO::2012-12-09 13:05:44,943::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2740::INFO::2012-12-09 13:05:44,944::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0157868862152', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2740::DEBUG::2012-12-09 13:05:44,944::task::1172::TaskManager.Task::(prepare) Task=`53e412f0-36e3-475f-8fdf-440b8dc33e45`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0157868862152', 'lastCheck': '1.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2740::DEBUG::2012-12-09 13:05:44,944::task::588::TaskManager.Task::(_updateState) Task=`53e412f0-36e3-475f-8fdf-440b8dc33e45`::moving from state preparing -> state finished
>Thread-2740::DEBUG::2012-12-09 13:05:44,944::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2740::DEBUG::2012-12-09 13:05:44,945::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2740::DEBUG::2012-12-09 13:05:44,945::task::978::TaskManager.Task::(_decref) Task=`53e412f0-36e3-475f-8fdf-440b8dc33e45`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:46,098::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:46,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015618 s, 65.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:48,147::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:48,187::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158315 s, 64.7 MB/s\n'; <rc> = 0
>Thread-2744::DEBUG::2012-12-09 13:05:50,008::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2744::DEBUG::2012-12-09 13:05:50,008::task::588::TaskManager.Task::(_updateState) Task=`e6aa0b80-68af-40c0-b23b-e75f5d0eba80`::moving from state init -> state preparing
>Thread-2744::INFO::2012-12-09 13:05:50,009::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2744::INFO::2012-12-09 13:05:50,009::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2744::DEBUG::2012-12-09 13:05:50,010::task::1172::TaskManager.Task::(prepare) Task=`e6aa0b80-68af-40c0-b23b-e75f5d0eba80`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2744::DEBUG::2012-12-09 13:05:50,010::task::588::TaskManager.Task::(_updateState) Task=`e6aa0b80-68af-40c0-b23b-e75f5d0eba80`::moving from state preparing -> state finished
>Thread-2744::DEBUG::2012-12-09 13:05:50,010::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2744::DEBUG::2012-12-09 13:05:50,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2744::DEBUG::2012-12-09 13:05:50,011::task::978::TaskManager.Task::(_decref) Task=`e6aa0b80-68af-40c0-b23b-e75f5d0eba80`::ref 0 aborting False
>Thread-2745::DEBUG::2012-12-09 13:05:50,024::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2745::DEBUG::2012-12-09 13:05:50,024::task::588::TaskManager.Task::(_updateState) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::moving from state init -> state preparing
>Thread-2745::INFO::2012-12-09 13:05:50,024::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2745::DEBUG::2012-12-09 13:05:50,025::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`db079d4e-d112-4a53-b909-137dc1689e4b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2745::DEBUG::2012-12-09 13:05:50,025::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2745::DEBUG::2012-12-09 13:05:50,026::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2745::DEBUG::2012-12-09 13:05:50,026::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`db079d4e-d112-4a53-b909-137dc1689e4b`::Granted request
>Thread-2745::DEBUG::2012-12-09 13:05:50,026::task::817::TaskManager.Task::(resourceAcquired) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2745::DEBUG::2012-12-09 13:05:50,027::task::978::TaskManager.Task::(_decref) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::ref 1 aborting False
>Thread-2745::DEBUG::2012-12-09 13:05:50,028::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2745::DEBUG::2012-12-09 13:05:50,028::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2745::DEBUG::2012-12-09 13:05:50,126::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2745::WARNING::2012-12-09 13:05:50,128::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2745::DEBUG::2012-12-09 13:05:50,128::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2745::WARNING::2012-12-09 13:05:50,128::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2745::INFO::2012-12-09 13:05:50,129::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2745::DEBUG::2012-12-09 13:05:50,129::task::1172::TaskManager.Task::(prepare) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2745::DEBUG::2012-12-09 13:05:50,129::task::588::TaskManager.Task::(_updateState) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::moving from state preparing -> state finished
>Thread-2745::DEBUG::2012-12-09 13:05:50,129::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2745::DEBUG::2012-12-09 13:05:50,130::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2745::DEBUG::2012-12-09 13:05:50,130::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2745::DEBUG::2012-12-09 13:05:50,130::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2745::DEBUG::2012-12-09 13:05:50,130::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2745::DEBUG::2012-12-09 13:05:50,130::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2745::DEBUG::2012-12-09 13:05:50,131::task::978::TaskManager.Task::(_decref) Task=`4f20254c-2a3f-41ef-b387-42827968ed49`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:50,195::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:50,233::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155639 s, 65.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:05:50,794::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:05:50,794::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:05:50,902::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:05:50,903::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:05:50,903::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:05:50,904::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:05:52,240::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:52,283::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160092 s, 64.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:05:53,089::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:05:53,098::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000598902 s, 6.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:54,290::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:54,330::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166559 s, 61.5 MB/s\n'; <rc> = 0
>Thread-2748::DEBUG::2012-12-09 13:05:55,314::task::588::TaskManager.Task::(_updateState) Task=`87a344f2-9cbd-4c72-894b-99229f795144`::moving from state init -> state preparing
>Thread-2748::INFO::2012-12-09 13:05:55,314::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2748::INFO::2012-12-09 13:05:55,314::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0100121498108', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-2748::DEBUG::2012-12-09 13:05:55,314::task::1172::TaskManager.Task::(prepare) Task=`87a344f2-9cbd-4c72-894b-99229f795144`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0100121498108', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-2748::DEBUG::2012-12-09 13:05:55,315::task::588::TaskManager.Task::(_updateState) Task=`87a344f2-9cbd-4c72-894b-99229f795144`::moving from state preparing -> state finished
>Thread-2748::DEBUG::2012-12-09 13:05:55,315::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2748::DEBUG::2012-12-09 13:05:55,315::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2748::DEBUG::2012-12-09 13:05:55,315::task::978::TaskManager.Task::(_decref) Task=`87a344f2-9cbd-4c72-894b-99229f795144`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:05:56,335::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:56,375::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160353 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:05:58,382::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:05:58,422::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162188 s, 63.1 MB/s\n'; <rc> = 0
>Thread-2752::DEBUG::2012-12-09 13:06:00,184::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2752::DEBUG::2012-12-09 13:06:00,184::task::588::TaskManager.Task::(_updateState) Task=`1376c481-fc55-488c-8e47-c11fc156c2bb`::moving from state init -> state preparing
>Thread-2752::INFO::2012-12-09 13:06:00,184::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2752::INFO::2012-12-09 13:06:00,185::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2752::DEBUG::2012-12-09 13:06:00,185::task::1172::TaskManager.Task::(prepare) Task=`1376c481-fc55-488c-8e47-c11fc156c2bb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2752::DEBUG::2012-12-09 13:06:00,185::task::588::TaskManager.Task::(_updateState) Task=`1376c481-fc55-488c-8e47-c11fc156c2bb`::moving from state preparing -> state finished
>Thread-2752::DEBUG::2012-12-09 13:06:00,186::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2752::DEBUG::2012-12-09 13:06:00,186::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2752::DEBUG::2012-12-09 13:06:00,186::task::978::TaskManager.Task::(_decref) Task=`1376c481-fc55-488c-8e47-c11fc156c2bb`::ref 0 aborting False
>Thread-2753::DEBUG::2012-12-09 13:06:00,201::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2753::DEBUG::2012-12-09 13:06:00,201::task::588::TaskManager.Task::(_updateState) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::moving from state init -> state preparing
>Thread-2753::INFO::2012-12-09 13:06:00,202::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2753::DEBUG::2012-12-09 13:06:00,202::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`295d41a5-5b57-475b-86b7-1d705268f762`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2753::DEBUG::2012-12-09 13:06:00,202::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2753::DEBUG::2012-12-09 13:06:00,202::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2753::DEBUG::2012-12-09 13:06:00,203::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`295d41a5-5b57-475b-86b7-1d705268f762`::Granted request
>Thread-2753::DEBUG::2012-12-09 13:06:00,203::task::817::TaskManager.Task::(resourceAcquired) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2753::DEBUG::2012-12-09 13:06:00,203::task::978::TaskManager.Task::(_decref) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::ref 1 aborting False
>Thread-2753::DEBUG::2012-12-09 13:06:00,204::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2753::DEBUG::2012-12-09 13:06:00,205::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2753::DEBUG::2012-12-09 13:06:00,306::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2753::WARNING::2012-12-09 13:06:00,307::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2753::DEBUG::2012-12-09 13:06:00,307::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2753::WARNING::2012-12-09 13:06:00,308::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2753::INFO::2012-12-09 13:06:00,308::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2753::DEBUG::2012-12-09 13:06:00,308::task::1172::TaskManager.Task::(prepare) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2753::DEBUG::2012-12-09 13:06:00,309::task::588::TaskManager.Task::(_updateState) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::moving from state preparing -> state finished
>Thread-2753::DEBUG::2012-12-09 13:06:00,309::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2753::DEBUG::2012-12-09 13:06:00,309::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2753::DEBUG::2012-12-09 13:06:00,309::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2753::DEBUG::2012-12-09 13:06:00,309::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2753::DEBUG::2012-12-09 13:06:00,310::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2753::DEBUG::2012-12-09 13:06:00,310::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2753::DEBUG::2012-12-09 13:06:00,310::task::978::TaskManager.Task::(_decref) Task=`3a64b7f7-5b5f-4ecd-9461-9ff5cd836a01`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:00,429::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:00,468::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164286 s, 62.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:06:00,905::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:00,905::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:06:01,006::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:01,007::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:01,008::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:01,008::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:06:02,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:02,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172102 s, 59.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:06:03,101::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:03,117::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000444069 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:04,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:04,564::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183829 s, 55.7 MB/s\n'; <rc> = 0
>Thread-2756::DEBUG::2012-12-09 13:06:06,015::task::588::TaskManager.Task::(_updateState) Task=`e782d24f-6227-4473-8daf-a8e1d6aeff66`::moving from state init -> state preparing
>Thread-2756::INFO::2012-12-09 13:06:06,015::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2756::INFO::2012-12-09 13:06:06,016::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162119865417', 'lastCheck': '2.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-2756::DEBUG::2012-12-09 13:06:06,016::task::1172::TaskManager.Task::(prepare) Task=`e782d24f-6227-4473-8daf-a8e1d6aeff66`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0162119865417', 'lastCheck': '2.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.0', 'code': 358, 'valid': False}}
>Thread-2756::DEBUG::2012-12-09 13:06:06,016::task::588::TaskManager.Task::(_updateState) Task=`e782d24f-6227-4473-8daf-a8e1d6aeff66`::moving from state preparing -> state finished
>Thread-2756::DEBUG::2012-12-09 13:06:06,016::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2756::DEBUG::2012-12-09 13:06:06,016::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2756::DEBUG::2012-12-09 13:06:06,017::task::978::TaskManager.Task::(_decref) Task=`e782d24f-6227-4473-8daf-a8e1d6aeff66`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:06,570::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:06,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169049 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:08,617::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:08,657::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164728 s, 62.2 MB/s\n'; <rc> = 0
>Thread-2760::DEBUG::2012-12-09 13:06:10,584::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2760::DEBUG::2012-12-09 13:06:10,585::task::588::TaskManager.Task::(_updateState) Task=`52ef05e7-a1fc-453a-82bd-968cb9650734`::moving from state init -> state preparing
>Thread-2760::INFO::2012-12-09 13:06:10,585::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2760::INFO::2012-12-09 13:06:10,586::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2760::DEBUG::2012-12-09 13:06:10,586::task::1172::TaskManager.Task::(prepare) Task=`52ef05e7-a1fc-453a-82bd-968cb9650734`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2760::DEBUG::2012-12-09 13:06:10,587::task::588::TaskManager.Task::(_updateState) Task=`52ef05e7-a1fc-453a-82bd-968cb9650734`::moving from state preparing -> state finished
>Thread-2760::DEBUG::2012-12-09 13:06:10,587::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2760::DEBUG::2012-12-09 13:06:10,588::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2760::DEBUG::2012-12-09 13:06:10,588::task::978::TaskManager.Task::(_decref) Task=`52ef05e7-a1fc-453a-82bd-968cb9650734`::ref 0 aborting False
>Thread-2761::DEBUG::2012-12-09 13:06:10,609::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2761::DEBUG::2012-12-09 13:06:10,610::task::588::TaskManager.Task::(_updateState) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::moving from state init -> state preparing
>Thread-2761::INFO::2012-12-09 13:06:10,610::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2761::DEBUG::2012-12-09 13:06:10,611::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`170388f0-dfc5-4ca3-9f2d-40f352ff1492`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2761::DEBUG::2012-12-09 13:06:10,612::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2761::DEBUG::2012-12-09 13:06:10,612::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2761::DEBUG::2012-12-09 13:06:10,613::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`170388f0-dfc5-4ca3-9f2d-40f352ff1492`::Granted request
>Thread-2761::DEBUG::2012-12-09 13:06:10,613::task::817::TaskManager.Task::(resourceAcquired) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2761::DEBUG::2012-12-09 13:06:10,614::task::978::TaskManager.Task::(_decref) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::ref 1 aborting False
>Thread-2761::DEBUG::2012-12-09 13:06:10,615::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2761::DEBUG::2012-12-09 13:06:10,616::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:10,664::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:10,701::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015612 s, 65.6 MB/s\n'; <rc> = 0
>Thread-2761::DEBUG::2012-12-09 13:06:10,724::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2761::WARNING::2012-12-09 13:06:10,725::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2761::DEBUG::2012-12-09 13:06:10,725::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2761::WARNING::2012-12-09 13:06:10,726::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2761::INFO::2012-12-09 13:06:10,727::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2761::DEBUG::2012-12-09 13:06:10,727::task::1172::TaskManager.Task::(prepare) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2761::DEBUG::2012-12-09 13:06:10,727::task::588::TaskManager.Task::(_updateState) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::moving from state preparing -> state finished
>Thread-2761::DEBUG::2012-12-09 13:06:10,728::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2761::DEBUG::2012-12-09 13:06:10,728::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2761::DEBUG::2012-12-09 13:06:10,729::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2761::DEBUG::2012-12-09 13:06:10,729::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2761::DEBUG::2012-12-09 13:06:10,729::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2761::DEBUG::2012-12-09 13:06:10,730::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2761::DEBUG::2012-12-09 13:06:10,730::task::978::TaskManager.Task::(_decref) Task=`c1ce6ec0-8b91-43dd-9be4-0c40e0b6b5e7`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:06:11,009::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:11,009::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:06:11,108::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:11,109::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:11,109::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:11,109::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:06:12,708::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:12,769::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279032 s, 36.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:06:13,120::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:13,129::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000493459 s, 8.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:14,776::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:14,816::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163103 s, 62.8 MB/s\n'; <rc> = 0
>Thread-2764::DEBUG::2012-12-09 13:06:16,309::task::588::TaskManager.Task::(_updateState) Task=`b35aaa6a-de8c-4a0f-b3da-6d27627e9368`::moving from state init -> state preparing
>Thread-2764::INFO::2012-12-09 13:06:16,309::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2764::INFO::2012-12-09 13:06:16,309::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0096709728241', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-2764::DEBUG::2012-12-09 13:06:16,310::task::1172::TaskManager.Task::(prepare) Task=`b35aaa6a-de8c-4a0f-b3da-6d27627e9368`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0096709728241', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-2764::DEBUG::2012-12-09 13:06:16,310::task::588::TaskManager.Task::(_updateState) Task=`b35aaa6a-de8c-4a0f-b3da-6d27627e9368`::moving from state preparing -> state finished
>Thread-2764::DEBUG::2012-12-09 13:06:16,310::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2764::DEBUG::2012-12-09 13:06:16,310::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2764::DEBUG::2012-12-09 13:06:16,310::task::978::TaskManager.Task::(_decref) Task=`b35aaa6a-de8c-4a0f-b3da-6d27627e9368`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:16,823::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:16,889::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0262191 s, 39.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:18,898::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:18,940::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171975 s, 59.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:20,947::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:20,999::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021441 s, 47.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:06:21,110::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:21,111::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2768::DEBUG::2012-12-09 13:06:21,119::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2768::DEBUG::2012-12-09 13:06:21,119::task::588::TaskManager.Task::(_updateState) Task=`13b6583b-eafc-4dbb-9ca0-12832ea6e46e`::moving from state init -> state preparing
>Thread-2768::INFO::2012-12-09 13:06:21,120::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2768::INFO::2012-12-09 13:06:21,120::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2768::DEBUG::2012-12-09 13:06:21,120::task::1172::TaskManager.Task::(prepare) Task=`13b6583b-eafc-4dbb-9ca0-12832ea6e46e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2768::DEBUG::2012-12-09 13:06:21,120::task::588::TaskManager.Task::(_updateState) Task=`13b6583b-eafc-4dbb-9ca0-12832ea6e46e`::moving from state preparing -> state finished
>Thread-2768::DEBUG::2012-12-09 13:06:21,120::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2768::DEBUG::2012-12-09 13:06:21,121::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2768::DEBUG::2012-12-09 13:06:21,121::task::978::TaskManager.Task::(_decref) Task=`13b6583b-eafc-4dbb-9ca0-12832ea6e46e`::ref 0 aborting False
>Thread-2769::DEBUG::2012-12-09 13:06:21,134::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2769::DEBUG::2012-12-09 13:06:21,135::task::588::TaskManager.Task::(_updateState) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::moving from state init -> state preparing
>Thread-2769::INFO::2012-12-09 13:06:21,135::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2769::DEBUG::2012-12-09 13:06:21,135::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`067ad517-2bc4-4918-a350-bb47c4e172e0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2769::DEBUG::2012-12-09 13:06:21,136::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2769::DEBUG::2012-12-09 13:06:21,136::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2769::DEBUG::2012-12-09 13:06:21,136::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`067ad517-2bc4-4918-a350-bb47c4e172e0`::Granted request
>Thread-2769::DEBUG::2012-12-09 13:06:21,137::task::817::TaskManager.Task::(resourceAcquired) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2769::DEBUG::2012-12-09 13:06:21,137::task::978::TaskManager.Task::(_decref) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 13:06:21,210::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:21,211::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:21,212::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:21,212::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2769::DEBUG::2012-12-09 13:06:21,213::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2769::DEBUG::2012-12-09 13:06:21,213::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2769::DEBUG::2012-12-09 13:06:21,310::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2769::WARNING::2012-12-09 13:06:21,312::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2769::DEBUG::2012-12-09 13:06:21,312::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2769::WARNING::2012-12-09 13:06:21,312::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2769::INFO::2012-12-09 13:06:21,313::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2769::DEBUG::2012-12-09 13:06:21,313::task::1172::TaskManager.Task::(prepare) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2769::DEBUG::2012-12-09 13:06:21,313::task::588::TaskManager.Task::(_updateState) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::moving from state preparing -> state finished
>Thread-2769::DEBUG::2012-12-09 13:06:21,313::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2769::DEBUG::2012-12-09 13:06:21,313::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2769::DEBUG::2012-12-09 13:06:21,314::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2769::DEBUG::2012-12-09 13:06:21,314::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2769::DEBUG::2012-12-09 13:06:21,314::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2769::DEBUG::2012-12-09 13:06:21,314::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2769::DEBUG::2012-12-09 13:06:21,315::task::978::TaskManager.Task::(_decref) Task=`1bd4b382-f348-4da9-b617-bd5596f2fbeb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:23,007::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:23,047::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164678 s, 62.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:06:23,132::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:23,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000488149 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:25,054::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:25,094::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161991 s, 63.2 MB/s\n'; <rc> = 0
>Thread-2772::DEBUG::2012-12-09 13:06:26,710::task::588::TaskManager.Task::(_updateState) Task=`c110ba89-c89e-4d71-a821-2e3184cd9ebb`::moving from state init -> state preparing
>Thread-2772::INFO::2012-12-09 13:06:26,710::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2772::INFO::2012-12-09 13:06:26,711::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850296020508', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-2772::DEBUG::2012-12-09 13:06:26,711::task::1172::TaskManager.Task::(prepare) Task=`c110ba89-c89e-4d71-a821-2e3184cd9ebb`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850296020508', 'lastCheck': '3.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-2772::DEBUG::2012-12-09 13:06:26,711::task::588::TaskManager.Task::(_updateState) Task=`c110ba89-c89e-4d71-a821-2e3184cd9ebb`::moving from state preparing -> state finished
>Thread-2772::DEBUG::2012-12-09 13:06:26,712::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2772::DEBUG::2012-12-09 13:06:26,712::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2772::DEBUG::2012-12-09 13:06:26,712::task::978::TaskManager.Task::(_decref) Task=`c110ba89-c89e-4d71-a821-2e3184cd9ebb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:27,101::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:27,143::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180104 s, 56.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:29,150::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:29,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021047 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:31,204::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:06:31,213::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:31,214::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:31,254::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247162 s, 41.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:06:31,321::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:31,322::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:31,322::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:31,323::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2776::DEBUG::2012-12-09 13:06:32,231::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2776::DEBUG::2012-12-09 13:06:32,232::task::588::TaskManager.Task::(_updateState) Task=`d8a32929-686e-4a7a-a446-6488c04373d9`::moving from state init -> state preparing
>Thread-2776::INFO::2012-12-09 13:06:32,232::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2776::INFO::2012-12-09 13:06:32,233::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2776::DEBUG::2012-12-09 13:06:32,233::task::1172::TaskManager.Task::(prepare) Task=`d8a32929-686e-4a7a-a446-6488c04373d9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2776::DEBUG::2012-12-09 13:06:32,233::task::588::TaskManager.Task::(_updateState) Task=`d8a32929-686e-4a7a-a446-6488c04373d9`::moving from state preparing -> state finished
>Thread-2776::DEBUG::2012-12-09 13:06:32,233::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2776::DEBUG::2012-12-09 13:06:32,233::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2776::DEBUG::2012-12-09 13:06:32,233::task::978::TaskManager.Task::(_decref) Task=`d8a32929-686e-4a7a-a446-6488c04373d9`::ref 0 aborting False
>Thread-2777::DEBUG::2012-12-09 13:06:32,246::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2777::DEBUG::2012-12-09 13:06:32,247::task::588::TaskManager.Task::(_updateState) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::moving from state init -> state preparing
>Thread-2777::INFO::2012-12-09 13:06:32,247::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2777::DEBUG::2012-12-09 13:06:32,247::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e98dd524-5877-4601-a215-268f3e343588`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2777::DEBUG::2012-12-09 13:06:32,247::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2777::DEBUG::2012-12-09 13:06:32,248::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2777::DEBUG::2012-12-09 13:06:32,248::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e98dd524-5877-4601-a215-268f3e343588`::Granted request
>Thread-2777::DEBUG::2012-12-09 13:06:32,248::task::817::TaskManager.Task::(resourceAcquired) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2777::DEBUG::2012-12-09 13:06:32,248::task::978::TaskManager.Task::(_decref) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::ref 1 aborting False
>Thread-2777::DEBUG::2012-12-09 13:06:32,249::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2777::DEBUG::2012-12-09 13:06:32,250::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2777::DEBUG::2012-12-09 13:06:32,348::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2777::WARNING::2012-12-09 13:06:32,349::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2777::DEBUG::2012-12-09 13:06:32,349::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2777::WARNING::2012-12-09 13:06:32,350::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2777::INFO::2012-12-09 13:06:32,350::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2777::DEBUG::2012-12-09 13:06:32,350::task::1172::TaskManager.Task::(prepare) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2777::DEBUG::2012-12-09 13:06:32,351::task::588::TaskManager.Task::(_updateState) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::moving from state preparing -> state finished
>Thread-2777::DEBUG::2012-12-09 13:06:32,351::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2777::DEBUG::2012-12-09 13:06:32,351::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2777::DEBUG::2012-12-09 13:06:32,351::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2777::DEBUG::2012-12-09 13:06:32,352::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2777::DEBUG::2012-12-09 13:06:32,352::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2777::DEBUG::2012-12-09 13:06:32,352::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2777::DEBUG::2012-12-09 13:06:32,352::task::978::TaskManager.Task::(_decref) Task=`ed0379f4-9838-40a1-a117-a0e7d458ea5b`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:06:33,143::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:33,167::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000411514 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:33,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:33,304::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194413 s, 52.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:35,311::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:35,365::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270495 s, 37.9 MB/s\n'; <rc> = 0
>Thread-2780::DEBUG::2012-12-09 13:06:37,088::task::588::TaskManager.Task::(_updateState) Task=`25cd8fff-835b-4feb-b2b0-01007a5c207a`::moving from state init -> state preparing
>Thread-2780::INFO::2012-12-09 13:06:37,089::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2780::INFO::2012-12-09 13:06:37,089::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0245721340179', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.8', 'code': 358, 'valid': False}}
>Thread-2780::DEBUG::2012-12-09 13:06:37,089::task::1172::TaskManager.Task::(prepare) Task=`25cd8fff-835b-4feb-b2b0-01007a5c207a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0245721340179', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.8', 'code': 358, 'valid': False}}
>Thread-2780::DEBUG::2012-12-09 13:06:37,089::task::588::TaskManager.Task::(_updateState) Task=`25cd8fff-835b-4feb-b2b0-01007a5c207a`::moving from state preparing -> state finished
>Thread-2780::DEBUG::2012-12-09 13:06:37,090::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2780::DEBUG::2012-12-09 13:06:37,090::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2780::DEBUG::2012-12-09 13:06:37,090::task::978::TaskManager.Task::(_decref) Task=`25cd8fff-835b-4feb-b2b0-01007a5c207a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:37,372::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:37,411::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162378 s, 63.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:39,419::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:39,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195326 s, 52.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:06:41,324::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:41,324::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:06:41,432::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:41,433::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:41,434::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:41,434::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:06:41,470::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:41,510::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169009 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2784::DEBUG::2012-12-09 13:06:42,847::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2784::DEBUG::2012-12-09 13:06:42,848::task::588::TaskManager.Task::(_updateState) Task=`bf4c2b7f-ee73-4828-bef6-02f523f611c1`::moving from state init -> state preparing
>Thread-2784::INFO::2012-12-09 13:06:42,849::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2784::INFO::2012-12-09 13:06:42,849::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2784::DEBUG::2012-12-09 13:06:42,849::task::1172::TaskManager.Task::(prepare) Task=`bf4c2b7f-ee73-4828-bef6-02f523f611c1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2784::DEBUG::2012-12-09 13:06:42,850::task::588::TaskManager.Task::(_updateState) Task=`bf4c2b7f-ee73-4828-bef6-02f523f611c1`::moving from state preparing -> state finished
>Thread-2784::DEBUG::2012-12-09 13:06:42,850::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2784::DEBUG::2012-12-09 13:06:42,850::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2784::DEBUG::2012-12-09 13:06:42,851::task::978::TaskManager.Task::(_decref) Task=`bf4c2b7f-ee73-4828-bef6-02f523f611c1`::ref 0 aborting False
>Thread-2785::DEBUG::2012-12-09 13:06:42,864::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2785::DEBUG::2012-12-09 13:06:42,864::task::588::TaskManager.Task::(_updateState) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::moving from state init -> state preparing
>Thread-2785::INFO::2012-12-09 13:06:42,865::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2785::DEBUG::2012-12-09 13:06:42,865::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f14f6c5b-6e66-4203-a9d5-f5039869030e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2785::DEBUG::2012-12-09 13:06:42,865::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2785::DEBUG::2012-12-09 13:06:42,865::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2785::DEBUG::2012-12-09 13:06:42,866::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f14f6c5b-6e66-4203-a9d5-f5039869030e`::Granted request
>Thread-2785::DEBUG::2012-12-09 13:06:42,866::task::817::TaskManager.Task::(resourceAcquired) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2785::DEBUG::2012-12-09 13:06:42,866::task::978::TaskManager.Task::(_decref) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::ref 1 aborting False
>Thread-2785::DEBUG::2012-12-09 13:06:42,867::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2785::DEBUG::2012-12-09 13:06:42,867::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2785::DEBUG::2012-12-09 13:06:42,965::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2785::WARNING::2012-12-09 13:06:42,966::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2785::DEBUG::2012-12-09 13:06:42,967::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2785::WARNING::2012-12-09 13:06:42,967::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2785::INFO::2012-12-09 13:06:42,968::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2785::DEBUG::2012-12-09 13:06:42,968::task::1172::TaskManager.Task::(prepare) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2785::DEBUG::2012-12-09 13:06:42,968::task::588::TaskManager.Task::(_updateState) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::moving from state preparing -> state finished
>Thread-2785::DEBUG::2012-12-09 13:06:42,968::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2785::DEBUG::2012-12-09 13:06:42,968::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2785::DEBUG::2012-12-09 13:06:42,969::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2785::DEBUG::2012-12-09 13:06:42,969::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2785::DEBUG::2012-12-09 13:06:42,969::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2785::DEBUG::2012-12-09 13:06:42,969::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2785::DEBUG::2012-12-09 13:06:42,970::task::978::TaskManager.Task::(_decref) Task=`7400cec6-73a1-4cbc-b53e-58d7eb6000e6`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:06:43,170::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:43,180::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00049759 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:43,517::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:43,558::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171949 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:45,564::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:45,605::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015495 s, 66.1 MB/s\n'; <rc> = 0
>Thread-2788::DEBUG::2012-12-09 13:06:47,416::task::588::TaskManager.Task::(_updateState) Task=`19020e03-2c79-44c0-a957-fac9abc80dc8`::moving from state init -> state preparing
>Thread-2788::INFO::2012-12-09 13:06:47,416::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2788::INFO::2012-12-09 13:06:47,416::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107820034027', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-2788::DEBUG::2012-12-09 13:06:47,416::task::1172::TaskManager.Task::(prepare) Task=`19020e03-2c79-44c0-a957-fac9abc80dc8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0107820034027', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-2788::DEBUG::2012-12-09 13:06:47,417::task::588::TaskManager.Task::(_updateState) Task=`19020e03-2c79-44c0-a957-fac9abc80dc8`::moving from state preparing -> state finished
>Thread-2788::DEBUG::2012-12-09 13:06:47,417::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2788::DEBUG::2012-12-09 13:06:47,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2788::DEBUG::2012-12-09 13:06:47,417::task::978::TaskManager.Task::(_decref) Task=`19020e03-2c79-44c0-a957-fac9abc80dc8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:47,612::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:47,652::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165266 s, 62.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:49,659::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:49,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173448 s, 59.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:06:51,435::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:06:51,436::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:06:51,536::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:06:51,537::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:06:51,538::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:06:51,538::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:06:51,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:51,745::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163076 s, 62.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:06:53,185::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:06:53,193::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000507367 s, 8.1 MB/s\n'; <rc> = 0
>Thread-2792::DEBUG::2012-12-09 13:06:53,297::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2792::DEBUG::2012-12-09 13:06:53,298::task::588::TaskManager.Task::(_updateState) Task=`b31c8221-6da5-44cb-87a5-007a6a6d2dab`::moving from state init -> state preparing
>Thread-2792::INFO::2012-12-09 13:06:53,298::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2792::INFO::2012-12-09 13:06:53,299::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2792::DEBUG::2012-12-09 13:06:53,299::task::1172::TaskManager.Task::(prepare) Task=`b31c8221-6da5-44cb-87a5-007a6a6d2dab`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2792::DEBUG::2012-12-09 13:06:53,299::task::588::TaskManager.Task::(_updateState) Task=`b31c8221-6da5-44cb-87a5-007a6a6d2dab`::moving from state preparing -> state finished
>Thread-2792::DEBUG::2012-12-09 13:06:53,299::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2792::DEBUG::2012-12-09 13:06:53,299::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2792::DEBUG::2012-12-09 13:06:53,300::task::978::TaskManager.Task::(_decref) Task=`b31c8221-6da5-44cb-87a5-007a6a6d2dab`::ref 0 aborting False
>Thread-2793::DEBUG::2012-12-09 13:06:53,314::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2793::DEBUG::2012-12-09 13:06:53,315::task::588::TaskManager.Task::(_updateState) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::moving from state init -> state preparing
>Thread-2793::INFO::2012-12-09 13:06:53,315::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2793::DEBUG::2012-12-09 13:06:53,315::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f6475e0a-b6af-42c7-ad10-d78fdd189c91`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2793::DEBUG::2012-12-09 13:06:53,316::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2793::DEBUG::2012-12-09 13:06:53,316::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2793::DEBUG::2012-12-09 13:06:53,316::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f6475e0a-b6af-42c7-ad10-d78fdd189c91`::Granted request
>Thread-2793::DEBUG::2012-12-09 13:06:53,316::task::817::TaskManager.Task::(resourceAcquired) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2793::DEBUG::2012-12-09 13:06:53,317::task::978::TaskManager.Task::(_decref) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::ref 1 aborting False
>Thread-2793::DEBUG::2012-12-09 13:06:53,317::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2793::DEBUG::2012-12-09 13:06:53,318::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2793::DEBUG::2012-12-09 13:06:53,420::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2793::WARNING::2012-12-09 13:06:53,421::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2793::DEBUG::2012-12-09 13:06:53,421::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2793::WARNING::2012-12-09 13:06:53,422::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2793::INFO::2012-12-09 13:06:53,422::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2793::DEBUG::2012-12-09 13:06:53,422::task::1172::TaskManager.Task::(prepare) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2793::DEBUG::2012-12-09 13:06:53,423::task::588::TaskManager.Task::(_updateState) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::moving from state preparing -> state finished
>Thread-2793::DEBUG::2012-12-09 13:06:53,423::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2793::DEBUG::2012-12-09 13:06:53,423::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2793::DEBUG::2012-12-09 13:06:53,423::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2793::DEBUG::2012-12-09 13:06:53,424::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2793::DEBUG::2012-12-09 13:06:53,424::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2793::DEBUG::2012-12-09 13:06:53,424::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2793::DEBUG::2012-12-09 13:06:53,424::task::978::TaskManager.Task::(_decref) Task=`2bed031d-e5cc-4111-b66b-f1b47825656d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:53,752::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:53,791::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163621 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:55,798::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:55,838::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165383 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:06:57,845::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:57,885::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016508 s, 62.0 MB/s\n'; <rc> = 0
>Thread-2796::DEBUG::2012-12-09 13:06:58,244::task::588::TaskManager.Task::(_updateState) Task=`7c1921db-ef40-4e24-813d-d091fbc44e46`::moving from state init -> state preparing
>Thread-2796::INFO::2012-12-09 13:06:58,245::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2796::INFO::2012-12-09 13:06:58,245::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00881290435791', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2796::DEBUG::2012-12-09 13:06:58,245::task::1172::TaskManager.Task::(prepare) Task=`7c1921db-ef40-4e24-813d-d091fbc44e46`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00881290435791', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False}}
>Thread-2796::DEBUG::2012-12-09 13:06:58,245::task::588::TaskManager.Task::(_updateState) Task=`7c1921db-ef40-4e24-813d-d091fbc44e46`::moving from state preparing -> state finished
>Thread-2796::DEBUG::2012-12-09 13:06:58,246::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2796::DEBUG::2012-12-09 13:06:58,246::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2796::DEBUG::2012-12-09 13:06:58,246::task::978::TaskManager.Task::(_decref) Task=`7c1921db-ef40-4e24-813d-d091fbc44e46`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:06:59,892::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:06:59,931::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165908 s, 61.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:01,539::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:01,540::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:01,638::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:01,640::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:01,640::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:01,641::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:01,938::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:01,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166923 s, 61.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:03,196::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:03,208::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000503569 s, 8.1 MB/s\n'; <rc> = 0
>Thread-2800::DEBUG::2012-12-09 13:07:03,671::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2800::DEBUG::2012-12-09 13:07:03,672::task::588::TaskManager.Task::(_updateState) Task=`1842fb8f-f939-40d4-813f-b4f36cf8a8ec`::moving from state init -> state preparing
>Thread-2800::INFO::2012-12-09 13:07:03,672::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2800::INFO::2012-12-09 13:07:03,673::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2800::DEBUG::2012-12-09 13:07:03,673::task::1172::TaskManager.Task::(prepare) Task=`1842fb8f-f939-40d4-813f-b4f36cf8a8ec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2800::DEBUG::2012-12-09 13:07:03,673::task::588::TaskManager.Task::(_updateState) Task=`1842fb8f-f939-40d4-813f-b4f36cf8a8ec`::moving from state preparing -> state finished
>Thread-2800::DEBUG::2012-12-09 13:07:03,673::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2800::DEBUG::2012-12-09 13:07:03,673::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2800::DEBUG::2012-12-09 13:07:03,674::task::978::TaskManager.Task::(_decref) Task=`1842fb8f-f939-40d4-813f-b4f36cf8a8ec`::ref 0 aborting False
>Thread-2801::DEBUG::2012-12-09 13:07:03,691::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2801::DEBUG::2012-12-09 13:07:03,691::task::588::TaskManager.Task::(_updateState) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::moving from state init -> state preparing
>Thread-2801::INFO::2012-12-09 13:07:03,692::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2801::DEBUG::2012-12-09 13:07:03,692::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cddcaa76-bf2b-4e5f-8f58-0c241da7d8b4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2801::DEBUG::2012-12-09 13:07:03,692::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2801::DEBUG::2012-12-09 13:07:03,692::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2801::DEBUG::2012-12-09 13:07:03,693::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cddcaa76-bf2b-4e5f-8f58-0c241da7d8b4`::Granted request
>Thread-2801::DEBUG::2012-12-09 13:07:03,693::task::817::TaskManager.Task::(resourceAcquired) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2801::DEBUG::2012-12-09 13:07:03,693::task::978::TaskManager.Task::(_decref) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::ref 1 aborting False
>Thread-2801::DEBUG::2012-12-09 13:07:03,694::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2801::DEBUG::2012-12-09 13:07:03,694::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2801::DEBUG::2012-12-09 13:07:03,791::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2801::WARNING::2012-12-09 13:07:03,792::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2801::DEBUG::2012-12-09 13:07:03,793::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2801::WARNING::2012-12-09 13:07:03,793::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2801::INFO::2012-12-09 13:07:03,793::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2801::DEBUG::2012-12-09 13:07:03,794::task::1172::TaskManager.Task::(prepare) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2801::DEBUG::2012-12-09 13:07:03,794::task::588::TaskManager.Task::(_updateState) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::moving from state preparing -> state finished
>Thread-2801::DEBUG::2012-12-09 13:07:03,794::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2801::DEBUG::2012-12-09 13:07:03,794::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2801::DEBUG::2012-12-09 13:07:03,795::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2801::DEBUG::2012-12-09 13:07:03,795::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2801::DEBUG::2012-12-09 13:07:03,795::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2801::DEBUG::2012-12-09 13:07:03,795::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2801::DEBUG::2012-12-09 13:07:03,795::task::978::TaskManager.Task::(_decref) Task=`8611658d-a9c9-46af-b1a8-116015fb3e89`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:03,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:04,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173971 s, 58.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:06,034::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:06,074::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164927 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:08,081::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:08,123::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161138 s, 63.5 MB/s\n'; <rc> = 0
>Thread-2804::DEBUG::2012-12-09 13:07:08,554::task::588::TaskManager.Task::(_updateState) Task=`8b17042d-a912-4efd-a0c1-93595d0faa2a`::moving from state init -> state preparing
>Thread-2804::INFO::2012-12-09 13:07:08,554::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2804::INFO::2012-12-09 13:07:08,555::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120959281921', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-2804::DEBUG::2012-12-09 13:07:08,555::task::1172::TaskManager.Task::(prepare) Task=`8b17042d-a912-4efd-a0c1-93595d0faa2a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120959281921', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.9', 'code': 358, 'valid': False}}
>Thread-2804::DEBUG::2012-12-09 13:07:08,555::task::588::TaskManager.Task::(_updateState) Task=`8b17042d-a912-4efd-a0c1-93595d0faa2a`::moving from state preparing -> state finished
>Thread-2804::DEBUG::2012-12-09 13:07:08,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2804::DEBUG::2012-12-09 13:07:08,555::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2804::DEBUG::2012-12-09 13:07:08,556::task::978::TaskManager.Task::(_decref) Task=`8b17042d-a912-4efd-a0c1-93595d0faa2a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:10,129::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:10,169::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158304 s, 64.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:11,642::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:11,642::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:11,740::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:11,741::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:11,741::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:11,742::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:12,176::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:12,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016542 s, 61.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:13,211::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:13,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00041791 s, 9.8 MB/s\n'; <rc> = 0
>Thread-2808::DEBUG::2012-12-09 13:07:13,860::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2808::DEBUG::2012-12-09 13:07:13,861::task::588::TaskManager.Task::(_updateState) Task=`17786d50-f13f-41a9-a7a5-cf147fb60e9e`::moving from state init -> state preparing
>Thread-2808::INFO::2012-12-09 13:07:13,861::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2808::INFO::2012-12-09 13:07:13,862::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2808::DEBUG::2012-12-09 13:07:13,862::task::1172::TaskManager.Task::(prepare) Task=`17786d50-f13f-41a9-a7a5-cf147fb60e9e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2808::DEBUG::2012-12-09 13:07:13,862::task::588::TaskManager.Task::(_updateState) Task=`17786d50-f13f-41a9-a7a5-cf147fb60e9e`::moving from state preparing -> state finished
>Thread-2808::DEBUG::2012-12-09 13:07:13,862::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2808::DEBUG::2012-12-09 13:07:13,862::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2808::DEBUG::2012-12-09 13:07:13,863::task::978::TaskManager.Task::(_decref) Task=`17786d50-f13f-41a9-a7a5-cf147fb60e9e`::ref 0 aborting False
>Thread-2809::DEBUG::2012-12-09 13:07:13,876::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2809::DEBUG::2012-12-09 13:07:13,877::task::588::TaskManager.Task::(_updateState) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::moving from state init -> state preparing
>Thread-2809::INFO::2012-12-09 13:07:13,877::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2809::DEBUG::2012-12-09 13:07:13,878::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b9f9069d-7264-4650-aaf0-4f3c87b61d51`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2809::DEBUG::2012-12-09 13:07:13,878::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2809::DEBUG::2012-12-09 13:07:13,878::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2809::DEBUG::2012-12-09 13:07:13,878::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b9f9069d-7264-4650-aaf0-4f3c87b61d51`::Granted request
>Thread-2809::DEBUG::2012-12-09 13:07:13,879::task::817::TaskManager.Task::(resourceAcquired) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2809::DEBUG::2012-12-09 13:07:13,879::task::978::TaskManager.Task::(_decref) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::ref 1 aborting False
>Thread-2809::DEBUG::2012-12-09 13:07:13,880::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2809::DEBUG::2012-12-09 13:07:13,880::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2809::DEBUG::2012-12-09 13:07:13,978::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2809::WARNING::2012-12-09 13:07:13,980::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2809::DEBUG::2012-12-09 13:07:13,980::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2809::WARNING::2012-12-09 13:07:13,980::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2809::INFO::2012-12-09 13:07:13,981::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2809::DEBUG::2012-12-09 13:07:13,981::task::1172::TaskManager.Task::(prepare) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2809::DEBUG::2012-12-09 13:07:13,981::task::588::TaskManager.Task::(_updateState) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::moving from state preparing -> state finished
>Thread-2809::DEBUG::2012-12-09 13:07:13,981::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2809::DEBUG::2012-12-09 13:07:13,981::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2809::DEBUG::2012-12-09 13:07:13,982::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2809::DEBUG::2012-12-09 13:07:13,982::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2809::DEBUG::2012-12-09 13:07:13,982::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2809::DEBUG::2012-12-09 13:07:13,982::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2809::DEBUG::2012-12-09 13:07:13,983::task::978::TaskManager.Task::(_decref) Task=`30b5393f-be8c-4217-8781-294ee663be2f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:14,223::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:14,262::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162485 s, 63.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:16,269::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:16,332::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263999 s, 38.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:18,341::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:18,382::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169918 s, 60.3 MB/s\n'; <rc> = 0
>Thread-2812::DEBUG::2012-12-09 13:07:18,839::task::588::TaskManager.Task::(_updateState) Task=`f51033f8-8e52-4e5a-98d2-ec799d5c12a5`::moving from state init -> state preparing
>Thread-2812::INFO::2012-12-09 13:07:18,840::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2812::INFO::2012-12-09 13:07:18,840::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.008633852005', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-2812::DEBUG::2012-12-09 13:07:18,840::task::1172::TaskManager.Task::(prepare) Task=`f51033f8-8e52-4e5a-98d2-ec799d5c12a5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.008633852005', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-2812::DEBUG::2012-12-09 13:07:18,840::task::588::TaskManager.Task::(_updateState) Task=`f51033f8-8e52-4e5a-98d2-ec799d5c12a5`::moving from state preparing -> state finished
>Thread-2812::DEBUG::2012-12-09 13:07:18,841::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2812::DEBUG::2012-12-09 13:07:18,841::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2812::DEBUG::2012-12-09 13:07:18,841::task::978::TaskManager.Task::(_decref) Task=`f51033f8-8e52-4e5a-98d2-ec799d5c12a5`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:20,388::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:20,428::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174353 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:21,743::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:21,743::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:21,843::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:21,844::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:21,844::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:21,845::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:22,435::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:22,475::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170084 s, 60.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:23,222::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:23,234::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000526762 s, 7.8 MB/s\n'; <rc> = 0
>Thread-2816::DEBUG::2012-12-09 13:07:24,037::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2816::DEBUG::2012-12-09 13:07:24,038::task::588::TaskManager.Task::(_updateState) Task=`e9f9c92a-3753-49a7-a917-b208a6f7b01f`::moving from state init -> state preparing
>Thread-2816::INFO::2012-12-09 13:07:24,038::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2816::INFO::2012-12-09 13:07:24,038::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2816::DEBUG::2012-12-09 13:07:24,039::task::1172::TaskManager.Task::(prepare) Task=`e9f9c92a-3753-49a7-a917-b208a6f7b01f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2816::DEBUG::2012-12-09 13:07:24,039::task::588::TaskManager.Task::(_updateState) Task=`e9f9c92a-3753-49a7-a917-b208a6f7b01f`::moving from state preparing -> state finished
>Thread-2816::DEBUG::2012-12-09 13:07:24,039::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2816::DEBUG::2012-12-09 13:07:24,039::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2816::DEBUG::2012-12-09 13:07:24,039::task::978::TaskManager.Task::(_decref) Task=`e9f9c92a-3753-49a7-a917-b208a6f7b01f`::ref 0 aborting False
>Thread-2817::DEBUG::2012-12-09 13:07:24,054::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2817::DEBUG::2012-12-09 13:07:24,054::task::588::TaskManager.Task::(_updateState) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::moving from state init -> state preparing
>Thread-2817::INFO::2012-12-09 13:07:24,055::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2817::DEBUG::2012-12-09 13:07:24,055::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`90ecee25-0569-4f2e-8d23-69a44d408c7e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2817::DEBUG::2012-12-09 13:07:24,055::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2817::DEBUG::2012-12-09 13:07:24,055::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2817::DEBUG::2012-12-09 13:07:24,056::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`90ecee25-0569-4f2e-8d23-69a44d408c7e`::Granted request
>Thread-2817::DEBUG::2012-12-09 13:07:24,056::task::817::TaskManager.Task::(resourceAcquired) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2817::DEBUG::2012-12-09 13:07:24,056::task::978::TaskManager.Task::(_decref) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::ref 1 aborting False
>Thread-2817::DEBUG::2012-12-09 13:07:24,057::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2817::DEBUG::2012-12-09 13:07:24,057::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2817::DEBUG::2012-12-09 13:07:24,164::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2817::WARNING::2012-12-09 13:07:24,165::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2817::DEBUG::2012-12-09 13:07:24,165::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2817::WARNING::2012-12-09 13:07:24,165::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2817::INFO::2012-12-09 13:07:24,166::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2817::DEBUG::2012-12-09 13:07:24,166::task::1172::TaskManager.Task::(prepare) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2817::DEBUG::2012-12-09 13:07:24,166::task::588::TaskManager.Task::(_updateState) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::moving from state preparing -> state finished
>Thread-2817::DEBUG::2012-12-09 13:07:24,166::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2817::DEBUG::2012-12-09 13:07:24,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2817::DEBUG::2012-12-09 13:07:24,167::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2817::DEBUG::2012-12-09 13:07:24,167::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2817::DEBUG::2012-12-09 13:07:24,167::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2817::DEBUG::2012-12-09 13:07:24,168::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2817::DEBUG::2012-12-09 13:07:24,168::task::978::TaskManager.Task::(_decref) Task=`c55d99bc-0f52-45d2-baac-9af9d80389ae`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:24,482::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:24,522::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160059 s, 64.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:26,529::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:26,569::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167009 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:28,576::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:28,617::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178659 s, 57.3 MB/s\n'; <rc> = 0
>Thread-2820::DEBUG::2012-12-09 13:07:29,600::task::588::TaskManager.Task::(_updateState) Task=`b6672832-db17-45e4-b085-7d6bca88ff42`::moving from state init -> state preparing
>Thread-2820::INFO::2012-12-09 13:07:29,601::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2820::INFO::2012-12-09 13:07:29,601::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122730731964', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.8', 'code': 358, 'valid': False}}
>Thread-2820::DEBUG::2012-12-09 13:07:29,602::task::1172::TaskManager.Task::(prepare) Task=`b6672832-db17-45e4-b085-7d6bca88ff42`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122730731964', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.8', 'code': 358, 'valid': False}}
>Thread-2820::DEBUG::2012-12-09 13:07:29,602::task::588::TaskManager.Task::(_updateState) Task=`b6672832-db17-45e4-b085-7d6bca88ff42`::moving from state preparing -> state finished
>Thread-2820::DEBUG::2012-12-09 13:07:29,603::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2820::DEBUG::2012-12-09 13:07:29,603::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2820::DEBUG::2012-12-09 13:07:29,603::task::978::TaskManager.Task::(_decref) Task=`b6672832-db17-45e4-b085-7d6bca88ff42`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:30,625::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:30,678::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273113 s, 37.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:31,846::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:31,846::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:31,946::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:31,948::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:31,948::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:31,948::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:32,685::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:32,725::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162281 s, 63.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:33,237::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:33,246::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000582815 s, 7.0 MB/s\n'; <rc> = 0
>Thread-2824::DEBUG::2012-12-09 13:07:34,565::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2824::DEBUG::2012-12-09 13:07:34,565::task::588::TaskManager.Task::(_updateState) Task=`ebfe54e7-070c-4ffb-a3aa-a09a9d520778`::moving from state init -> state preparing
>Thread-2824::INFO::2012-12-09 13:07:34,566::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2824::INFO::2012-12-09 13:07:34,566::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2824::DEBUG::2012-12-09 13:07:34,566::task::1172::TaskManager.Task::(prepare) Task=`ebfe54e7-070c-4ffb-a3aa-a09a9d520778`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2824::DEBUG::2012-12-09 13:07:34,566::task::588::TaskManager.Task::(_updateState) Task=`ebfe54e7-070c-4ffb-a3aa-a09a9d520778`::moving from state preparing -> state finished
>Thread-2824::DEBUG::2012-12-09 13:07:34,566::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2824::DEBUG::2012-12-09 13:07:34,567::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2824::DEBUG::2012-12-09 13:07:34,567::task::978::TaskManager.Task::(_decref) Task=`ebfe54e7-070c-4ffb-a3aa-a09a9d520778`::ref 0 aborting False
>Thread-2825::DEBUG::2012-12-09 13:07:34,586::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2825::DEBUG::2012-12-09 13:07:34,587::task::588::TaskManager.Task::(_updateState) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::moving from state init -> state preparing
>Thread-2825::INFO::2012-12-09 13:07:34,587::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2825::DEBUG::2012-12-09 13:07:34,588::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cb3596ff-41df-4ce7-be08-08ab804a2567`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2825::DEBUG::2012-12-09 13:07:34,588::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2825::DEBUG::2012-12-09 13:07:34,588::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2825::DEBUG::2012-12-09 13:07:34,589::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`cb3596ff-41df-4ce7-be08-08ab804a2567`::Granted request
>Thread-2825::DEBUG::2012-12-09 13:07:34,589::task::817::TaskManager.Task::(resourceAcquired) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2825::DEBUG::2012-12-09 13:07:34,589::task::978::TaskManager.Task::(_decref) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::ref 1 aborting False
>Thread-2825::DEBUG::2012-12-09 13:07:34,590::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2825::DEBUG::2012-12-09 13:07:34,591::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2825::DEBUG::2012-12-09 13:07:34,691::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2825::WARNING::2012-12-09 13:07:34,692::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2825::DEBUG::2012-12-09 13:07:34,692::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2825::WARNING::2012-12-09 13:07:34,693::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2825::INFO::2012-12-09 13:07:34,693::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2825::DEBUG::2012-12-09 13:07:34,693::task::1172::TaskManager.Task::(prepare) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2825::DEBUG::2012-12-09 13:07:34,694::task::588::TaskManager.Task::(_updateState) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::moving from state preparing -> state finished
>Thread-2825::DEBUG::2012-12-09 13:07:34,694::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2825::DEBUG::2012-12-09 13:07:34,694::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2825::DEBUG::2012-12-09 13:07:34,694::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2825::DEBUG::2012-12-09 13:07:34,694::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2825::DEBUG::2012-12-09 13:07:34,695::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2825::DEBUG::2012-12-09 13:07:34,695::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2825::DEBUG::2012-12-09 13:07:34,695::task::978::TaskManager.Task::(_decref) Task=`417b7574-988a-42e6-82c4-a26df8039b96`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:34,731::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:34,771::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167934 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:36,778::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:36,820::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164805 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:38,826::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:38,866::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164918 s, 62.1 MB/s\n'; <rc> = 0
>Thread-2828::DEBUG::2012-12-09 13:07:40,205::task::588::TaskManager.Task::(_updateState) Task=`5bb43372-cef4-49ce-acd0-7168ece4442a`::moving from state init -> state preparing
>Thread-2828::INFO::2012-12-09 13:07:40,206::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2828::INFO::2012-12-09 13:07:40,206::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00924181938171', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.3', 'code': 358, 'valid': False}}
>Thread-2828::DEBUG::2012-12-09 13:07:40,207::task::1172::TaskManager.Task::(prepare) Task=`5bb43372-cef4-49ce-acd0-7168ece4442a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00924181938171', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.3', 'code': 358, 'valid': False}}
>Thread-2828::DEBUG::2012-12-09 13:07:40,207::task::588::TaskManager.Task::(_updateState) Task=`5bb43372-cef4-49ce-acd0-7168ece4442a`::moving from state preparing -> state finished
>Thread-2828::DEBUG::2012-12-09 13:07:40,208::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2828::DEBUG::2012-12-09 13:07:40,208::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2828::DEBUG::2012-12-09 13:07:40,209::task::978::TaskManager.Task::(_decref) Task=`5bb43372-cef4-49ce-acd0-7168ece4442a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:40,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:40,933::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0232775 s, 44.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:41,949::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:41,950::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:42,055::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:42,056::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:42,056::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:42,056::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:42,943::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:42,983::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174779 s, 58.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:43,250::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:43,262::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000793818 s, 5.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:44,990::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:45,031::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183309 s, 55.9 MB/s\n'; <rc> = 0
>Thread-2832::DEBUG::2012-12-09 13:07:46,386::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2832::DEBUG::2012-12-09 13:07:46,387::task::588::TaskManager.Task::(_updateState) Task=`c512ed84-eb05-4e7f-9423-951ce09390c8`::moving from state init -> state preparing
>Thread-2832::INFO::2012-12-09 13:07:46,387::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2832::INFO::2012-12-09 13:07:46,388::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2832::DEBUG::2012-12-09 13:07:46,389::task::1172::TaskManager.Task::(prepare) Task=`c512ed84-eb05-4e7f-9423-951ce09390c8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2832::DEBUG::2012-12-09 13:07:46,389::task::588::TaskManager.Task::(_updateState) Task=`c512ed84-eb05-4e7f-9423-951ce09390c8`::moving from state preparing -> state finished
>Thread-2832::DEBUG::2012-12-09 13:07:46,390::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2832::DEBUG::2012-12-09 13:07:46,391::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2832::DEBUG::2012-12-09 13:07:46,391::task::978::TaskManager.Task::(_decref) Task=`c512ed84-eb05-4e7f-9423-951ce09390c8`::ref 0 aborting False
>Thread-2833::DEBUG::2012-12-09 13:07:46,405::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2833::DEBUG::2012-12-09 13:07:46,405::task::588::TaskManager.Task::(_updateState) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::moving from state init -> state preparing
>Thread-2833::INFO::2012-12-09 13:07:46,406::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2833::DEBUG::2012-12-09 13:07:46,406::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`24470a7e-924c-4f66-8a2a-f3b1f996f0ba`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2833::DEBUG::2012-12-09 13:07:46,406::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2833::DEBUG::2012-12-09 13:07:46,406::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2833::DEBUG::2012-12-09 13:07:46,407::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`24470a7e-924c-4f66-8a2a-f3b1f996f0ba`::Granted request
>Thread-2833::DEBUG::2012-12-09 13:07:46,407::task::817::TaskManager.Task::(resourceAcquired) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2833::DEBUG::2012-12-09 13:07:46,407::task::978::TaskManager.Task::(_decref) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::ref 1 aborting False
>Thread-2833::DEBUG::2012-12-09 13:07:46,408::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2833::DEBUG::2012-12-09 13:07:46,409::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2833::DEBUG::2012-12-09 13:07:46,509::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2833::WARNING::2012-12-09 13:07:46,510::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2833::DEBUG::2012-12-09 13:07:46,510::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2833::WARNING::2012-12-09 13:07:46,510::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2833::INFO::2012-12-09 13:07:46,511::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2833::DEBUG::2012-12-09 13:07:46,511::task::1172::TaskManager.Task::(prepare) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2833::DEBUG::2012-12-09 13:07:46,511::task::588::TaskManager.Task::(_updateState) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::moving from state preparing -> state finished
>Thread-2833::DEBUG::2012-12-09 13:07:46,511::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2833::DEBUG::2012-12-09 13:07:46,512::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2833::DEBUG::2012-12-09 13:07:46,512::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2833::DEBUG::2012-12-09 13:07:46,512::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2833::DEBUG::2012-12-09 13:07:46,512::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2833::DEBUG::2012-12-09 13:07:46,512::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2833::DEBUG::2012-12-09 13:07:46,513::task::978::TaskManager.Task::(_decref) Task=`48f50371-f2dc-49e0-a438-4782fca749d7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:47,039::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:47,081::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179151 s, 57.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:49,088::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:49,128::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160305 s, 63.9 MB/s\n'; <rc> = 0
>Thread-2836::DEBUG::2012-12-09 13:07:50,722::task::588::TaskManager.Task::(_updateState) Task=`d752ccce-833e-4a71-b6ad-2ed414cd58cf`::moving from state init -> state preparing
>Thread-2836::INFO::2012-12-09 13:07:50,722::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2836::INFO::2012-12-09 13:07:50,723::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0131270885468', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-2836::DEBUG::2012-12-09 13:07:50,723::task::1172::TaskManager.Task::(prepare) Task=`d752ccce-833e-4a71-b6ad-2ed414cd58cf`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0131270885468', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.7', 'code': 358, 'valid': False}}
>Thread-2836::DEBUG::2012-12-09 13:07:50,723::task::588::TaskManager.Task::(_updateState) Task=`d752ccce-833e-4a71-b6ad-2ed414cd58cf`::moving from state preparing -> state finished
>Thread-2836::DEBUG::2012-12-09 13:07:50,723::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2836::DEBUG::2012-12-09 13:07:50,723::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2836::DEBUG::2012-12-09 13:07:50,724::task::978::TaskManager.Task::(_decref) Task=`d752ccce-833e-4a71-b6ad-2ed414cd58cf`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:51,134::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:51,183::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239473 s, 42.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:07:52,057::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:07:52,058::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:07:52,162::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:07:52,163::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:07:52,164::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:07:52,164::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:07:53,190::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:53,230::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163774 s, 62.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:07:53,265::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:07:53,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000557928 s, 7.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:55,237::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:55,277::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165581 s, 61.8 MB/s\n'; <rc> = 0
>Thread-2840::DEBUG::2012-12-09 13:07:56,564::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2840::DEBUG::2012-12-09 13:07:56,565::task::588::TaskManager.Task::(_updateState) Task=`30eb42c9-012b-4243-97d3-7706fe4d4a0b`::moving from state init -> state preparing
>Thread-2840::INFO::2012-12-09 13:07:56,565::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2840::INFO::2012-12-09 13:07:56,565::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2840::DEBUG::2012-12-09 13:07:56,565::task::1172::TaskManager.Task::(prepare) Task=`30eb42c9-012b-4243-97d3-7706fe4d4a0b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2840::DEBUG::2012-12-09 13:07:56,566::task::588::TaskManager.Task::(_updateState) Task=`30eb42c9-012b-4243-97d3-7706fe4d4a0b`::moving from state preparing -> state finished
>Thread-2840::DEBUG::2012-12-09 13:07:56,566::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2840::DEBUG::2012-12-09 13:07:56,566::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2840::DEBUG::2012-12-09 13:07:56,566::task::978::TaskManager.Task::(_decref) Task=`30eb42c9-012b-4243-97d3-7706fe4d4a0b`::ref 0 aborting False
>Thread-2841::DEBUG::2012-12-09 13:07:56,579::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2841::DEBUG::2012-12-09 13:07:56,579::task::588::TaskManager.Task::(_updateState) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::moving from state init -> state preparing
>Thread-2841::INFO::2012-12-09 13:07:56,579::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2841::DEBUG::2012-12-09 13:07:56,580::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`488a40df-80ad-4e5e-9607-da33dbfad20d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2841::DEBUG::2012-12-09 13:07:56,580::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2841::DEBUG::2012-12-09 13:07:56,580::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2841::DEBUG::2012-12-09 13:07:56,580::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`488a40df-80ad-4e5e-9607-da33dbfad20d`::Granted request
>Thread-2841::DEBUG::2012-12-09 13:07:56,581::task::817::TaskManager.Task::(resourceAcquired) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2841::DEBUG::2012-12-09 13:07:56,581::task::978::TaskManager.Task::(_decref) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::ref 1 aborting False
>Thread-2841::DEBUG::2012-12-09 13:07:56,582::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2841::DEBUG::2012-12-09 13:07:56,582::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2841::DEBUG::2012-12-09 13:07:56,681::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2841::WARNING::2012-12-09 13:07:56,683::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2841::DEBUG::2012-12-09 13:07:56,683::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2841::WARNING::2012-12-09 13:07:56,683::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2841::INFO::2012-12-09 13:07:56,684::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2841::DEBUG::2012-12-09 13:07:56,684::task::1172::TaskManager.Task::(prepare) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2841::DEBUG::2012-12-09 13:07:56,684::task::588::TaskManager.Task::(_updateState) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::moving from state preparing -> state finished
>Thread-2841::DEBUG::2012-12-09 13:07:56,684::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2841::DEBUG::2012-12-09 13:07:56,685::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2841::DEBUG::2012-12-09 13:07:56,685::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2841::DEBUG::2012-12-09 13:07:56,685::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2841::DEBUG::2012-12-09 13:07:56,685::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2841::DEBUG::2012-12-09 13:07:56,686::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2841::DEBUG::2012-12-09 13:07:56,686::task::978::TaskManager.Task::(_decref) Task=`331e541d-4e4a-4064-8c20-949f54857ab9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:07:57,284::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:57,326::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170921 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:07:59,333::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:07:59,374::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172026 s, 59.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:01,380::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:01,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163028 s, 62.8 MB/s\n'; <rc> = 0
>Thread-2844::DEBUG::2012-12-09 13:08:02,067::task::588::TaskManager.Task::(_updateState) Task=`40241c09-b4b4-4c8b-a3b8-5f1c6ce83579`::moving from state init -> state preparing
>Thread-2844::INFO::2012-12-09 13:08:02,067::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2844::INFO::2012-12-09 13:08:02,067::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00942301750183', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2844::DEBUG::2012-12-09 13:08:02,068::task::1172::TaskManager.Task::(prepare) Task=`40241c09-b4b4-4c8b-a3b8-5f1c6ce83579`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00942301750183', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-2844::DEBUG::2012-12-09 13:08:02,068::task::588::TaskManager.Task::(_updateState) Task=`40241c09-b4b4-4c8b-a3b8-5f1c6ce83579`::moving from state preparing -> state finished
>Thread-2844::DEBUG::2012-12-09 13:08:02,068::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2844::DEBUG::2012-12-09 13:08:02,068::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2844::DEBUG::2012-12-09 13:08:02,068::task::978::TaskManager.Task::(_decref) Task=`40241c09-b4b4-4c8b-a3b8-5f1c6ce83579`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:08:02,165::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:02,166::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:02,264::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:02,265::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:02,266::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:02,266::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:08:03,277::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:03,286::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000601357 s, 6.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:03,427::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:03,469::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164703 s, 62.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:05,475::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:05,516::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168943 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2848::DEBUG::2012-12-09 13:08:07,017::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2848::DEBUG::2012-12-09 13:08:07,018::task::588::TaskManager.Task::(_updateState) Task=`a3611f6e-e7a7-4d2b-a9a5-a1a69ccf5c50`::moving from state init -> state preparing
>Thread-2848::INFO::2012-12-09 13:08:07,018::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2848::INFO::2012-12-09 13:08:07,019::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2848::DEBUG::2012-12-09 13:08:07,019::task::1172::TaskManager.Task::(prepare) Task=`a3611f6e-e7a7-4d2b-a9a5-a1a69ccf5c50`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2848::DEBUG::2012-12-09 13:08:07,019::task::588::TaskManager.Task::(_updateState) Task=`a3611f6e-e7a7-4d2b-a9a5-a1a69ccf5c50`::moving from state preparing -> state finished
>Thread-2848::DEBUG::2012-12-09 13:08:07,019::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2848::DEBUG::2012-12-09 13:08:07,020::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2848::DEBUG::2012-12-09 13:08:07,020::task::978::TaskManager.Task::(_decref) Task=`a3611f6e-e7a7-4d2b-a9a5-a1a69ccf5c50`::ref 0 aborting False
>Thread-2849::DEBUG::2012-12-09 13:08:07,035::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2849::DEBUG::2012-12-09 13:08:07,035::task::588::TaskManager.Task::(_updateState) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::moving from state init -> state preparing
>Thread-2849::INFO::2012-12-09 13:08:07,035::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2849::DEBUG::2012-12-09 13:08:07,036::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51a52768-1f98-4e21-8fcd-583bc2a5bd70`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2849::DEBUG::2012-12-09 13:08:07,036::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2849::DEBUG::2012-12-09 13:08:07,036::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2849::DEBUG::2012-12-09 13:08:07,036::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51a52768-1f98-4e21-8fcd-583bc2a5bd70`::Granted request
>Thread-2849::DEBUG::2012-12-09 13:08:07,037::task::817::TaskManager.Task::(resourceAcquired) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2849::DEBUG::2012-12-09 13:08:07,037::task::978::TaskManager.Task::(_decref) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::ref 1 aborting False
>Thread-2849::DEBUG::2012-12-09 13:08:07,038::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2849::DEBUG::2012-12-09 13:08:07,038::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2849::DEBUG::2012-12-09 13:08:07,136::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2849::WARNING::2012-12-09 13:08:07,138::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2849::DEBUG::2012-12-09 13:08:07,138::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2849::WARNING::2012-12-09 13:08:07,138::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2849::INFO::2012-12-09 13:08:07,139::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2849::DEBUG::2012-12-09 13:08:07,139::task::1172::TaskManager.Task::(prepare) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2849::DEBUG::2012-12-09 13:08:07,139::task::588::TaskManager.Task::(_updateState) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::moving from state preparing -> state finished
>Thread-2849::DEBUG::2012-12-09 13:08:07,139::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2849::DEBUG::2012-12-09 13:08:07,140::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2849::DEBUG::2012-12-09 13:08:07,140::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2849::DEBUG::2012-12-09 13:08:07,140::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2849::DEBUG::2012-12-09 13:08:07,140::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2849::DEBUG::2012-12-09 13:08:07,140::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2849::DEBUG::2012-12-09 13:08:07,141::task::978::TaskManager.Task::(_decref) Task=`a77b3c6c-57e9-4ed7-ad19-6138617d883c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:07,523::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:07,565::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174145 s, 58.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:09,572::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:09,612::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164202 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:11,619::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:11,659::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171658 s, 59.7 MB/s\n'; <rc> = 0
>Thread-2852::DEBUG::2012-12-09 13:08:12,254::task::588::TaskManager.Task::(_updateState) Task=`a90dd258-d3cb-47fd-879b-476c24162284`::moving from state init -> state preparing
>Thread-2852::INFO::2012-12-09 13:08:12,255::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2852::INFO::2012-12-09 13:08:12,255::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00942301750183', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-2852::DEBUG::2012-12-09 13:08:12,255::task::1172::TaskManager.Task::(prepare) Task=`a90dd258-d3cb-47fd-879b-476c24162284`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00942301750183', 'lastCheck': '9.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-2852::DEBUG::2012-12-09 13:08:12,256::task::588::TaskManager.Task::(_updateState) Task=`a90dd258-d3cb-47fd-879b-476c24162284`::moving from state preparing -> state finished
>Thread-2852::DEBUG::2012-12-09 13:08:12,256::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2852::DEBUG::2012-12-09 13:08:12,256::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2852::DEBUG::2012-12-09 13:08:12,257::task::978::TaskManager.Task::(_decref) Task=`a90dd258-d3cb-47fd-879b-476c24162284`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:08:12,267::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:12,268::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:12,382::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:12,383::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:12,384::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:12,385::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:08:13,289::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:13,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000550366 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:13,666::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:13,707::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166548 s, 61.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:15,713::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:15,754::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167635 s, 61.1 MB/s\n'; <rc> = 0
>Thread-2856::DEBUG::2012-12-09 13:08:17,415::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2856::DEBUG::2012-12-09 13:08:17,415::task::588::TaskManager.Task::(_updateState) Task=`a520becb-8f00-4ae4-b5b2-3199f221d3fd`::moving from state init -> state preparing
>Thread-2856::INFO::2012-12-09 13:08:17,415::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2856::INFO::2012-12-09 13:08:17,416::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2856::DEBUG::2012-12-09 13:08:17,416::task::1172::TaskManager.Task::(prepare) Task=`a520becb-8f00-4ae4-b5b2-3199f221d3fd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2856::DEBUG::2012-12-09 13:08:17,416::task::588::TaskManager.Task::(_updateState) Task=`a520becb-8f00-4ae4-b5b2-3199f221d3fd`::moving from state preparing -> state finished
>Thread-2856::DEBUG::2012-12-09 13:08:17,416::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2856::DEBUG::2012-12-09 13:08:17,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2856::DEBUG::2012-12-09 13:08:17,417::task::978::TaskManager.Task::(_decref) Task=`a520becb-8f00-4ae4-b5b2-3199f221d3fd`::ref 0 aborting False
>Thread-2857::DEBUG::2012-12-09 13:08:17,429::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2857::DEBUG::2012-12-09 13:08:17,430::task::588::TaskManager.Task::(_updateState) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::moving from state init -> state preparing
>Thread-2857::INFO::2012-12-09 13:08:17,430::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2857::DEBUG::2012-12-09 13:08:17,430::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`05a1487d-e957-4cad-9b0a-731b45d32a5d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2857::DEBUG::2012-12-09 13:08:17,430::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2857::DEBUG::2012-12-09 13:08:17,431::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2857::DEBUG::2012-12-09 13:08:17,431::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`05a1487d-e957-4cad-9b0a-731b45d32a5d`::Granted request
>Thread-2857::DEBUG::2012-12-09 13:08:17,431::task::817::TaskManager.Task::(resourceAcquired) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2857::DEBUG::2012-12-09 13:08:17,431::task::978::TaskManager.Task::(_decref) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::ref 1 aborting False
>Thread-2857::DEBUG::2012-12-09 13:08:17,432::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2857::DEBUG::2012-12-09 13:08:17,433::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2857::DEBUG::2012-12-09 13:08:17,530::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2857::WARNING::2012-12-09 13:08:17,531::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2857::DEBUG::2012-12-09 13:08:17,531::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2857::WARNING::2012-12-09 13:08:17,532::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2857::INFO::2012-12-09 13:08:17,532::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2857::DEBUG::2012-12-09 13:08:17,532::task::1172::TaskManager.Task::(prepare) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2857::DEBUG::2012-12-09 13:08:17,532::task::588::TaskManager.Task::(_updateState) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::moving from state preparing -> state finished
>Thread-2857::DEBUG::2012-12-09 13:08:17,533::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2857::DEBUG::2012-12-09 13:08:17,533::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2857::DEBUG::2012-12-09 13:08:17,533::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2857::DEBUG::2012-12-09 13:08:17,533::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2857::DEBUG::2012-12-09 13:08:17,534::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2857::DEBUG::2012-12-09 13:08:17,534::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2857::DEBUG::2012-12-09 13:08:17,534::task::978::TaskManager.Task::(_decref) Task=`63f83df6-70a2-4492-a339-219eef0f071c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:17,761::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:17,801::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016418 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:19,807::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:19,848::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164277 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:21,855::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:21,903::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237411 s, 43.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:08:22,386::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:22,386::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:22,496::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:22,497::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:22,497::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:22,498::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2860::DEBUG::2012-12-09 13:08:22,543::task::588::TaskManager.Task::(_updateState) Task=`545398df-bfc2-42fd-b01b-40d62f3803ab`::moving from state init -> state preparing
>Thread-2860::INFO::2012-12-09 13:08:22,544::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2860::INFO::2012-12-09 13:08:22,544::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263390541077', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-2860::DEBUG::2012-12-09 13:08:22,544::task::1172::TaskManager.Task::(prepare) Task=`545398df-bfc2-42fd-b01b-40d62f3803ab`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0263390541077', 'lastCheck': '9.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-2860::DEBUG::2012-12-09 13:08:22,544::task::588::TaskManager.Task::(_updateState) Task=`545398df-bfc2-42fd-b01b-40d62f3803ab`::moving from state preparing -> state finished
>Thread-2860::DEBUG::2012-12-09 13:08:22,544::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2860::DEBUG::2012-12-09 13:08:22,545::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2860::DEBUG::2012-12-09 13:08:22,545::task::978::TaskManager.Task::(_decref) Task=`545398df-bfc2-42fd-b01b-40d62f3803ab`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:08:23,318::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:23,330::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000526244 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:23,910::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:23,950::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166499 s, 61.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:25,957::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:25,997::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167075 s, 61.3 MB/s\n'; <rc> = 0
>Thread-2864::DEBUG::2012-12-09 13:08:27,737::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2864::DEBUG::2012-12-09 13:08:27,738::task::588::TaskManager.Task::(_updateState) Task=`4a9d7472-8ac3-4e59-b9fc-65010a1183f3`::moving from state init -> state preparing
>Thread-2864::INFO::2012-12-09 13:08:27,738::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2864::INFO::2012-12-09 13:08:27,739::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2864::DEBUG::2012-12-09 13:08:27,739::task::1172::TaskManager.Task::(prepare) Task=`4a9d7472-8ac3-4e59-b9fc-65010a1183f3`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2864::DEBUG::2012-12-09 13:08:27,739::task::588::TaskManager.Task::(_updateState) Task=`4a9d7472-8ac3-4e59-b9fc-65010a1183f3`::moving from state preparing -> state finished
>Thread-2864::DEBUG::2012-12-09 13:08:27,739::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2864::DEBUG::2012-12-09 13:08:27,739::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2864::DEBUG::2012-12-09 13:08:27,740::task::978::TaskManager.Task::(_decref) Task=`4a9d7472-8ac3-4e59-b9fc-65010a1183f3`::ref 0 aborting False
>Thread-2865::DEBUG::2012-12-09 13:08:27,753::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2865::DEBUG::2012-12-09 13:08:27,754::task::588::TaskManager.Task::(_updateState) Task=`0381f171-af28-4669-bd2e-f07526197d66`::moving from state init -> state preparing
>Thread-2865::INFO::2012-12-09 13:08:27,754::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2865::DEBUG::2012-12-09 13:08:27,755::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e2c551ff-a577-4f9a-9173-e9aca0dd6dcd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2865::DEBUG::2012-12-09 13:08:27,755::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2865::DEBUG::2012-12-09 13:08:27,755::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2865::DEBUG::2012-12-09 13:08:27,755::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e2c551ff-a577-4f9a-9173-e9aca0dd6dcd`::Granted request
>Thread-2865::DEBUG::2012-12-09 13:08:27,756::task::817::TaskManager.Task::(resourceAcquired) Task=`0381f171-af28-4669-bd2e-f07526197d66`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2865::DEBUG::2012-12-09 13:08:27,756::task::978::TaskManager.Task::(_decref) Task=`0381f171-af28-4669-bd2e-f07526197d66`::ref 1 aborting False
>Thread-2865::DEBUG::2012-12-09 13:08:27,757::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2865::DEBUG::2012-12-09 13:08:27,757::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2865::DEBUG::2012-12-09 13:08:27,857::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2865::WARNING::2012-12-09 13:08:27,858::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2865::DEBUG::2012-12-09 13:08:27,858::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2865::WARNING::2012-12-09 13:08:27,859::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2865::INFO::2012-12-09 13:08:27,859::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2865::DEBUG::2012-12-09 13:08:27,859::task::1172::TaskManager.Task::(prepare) Task=`0381f171-af28-4669-bd2e-f07526197d66`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2865::DEBUG::2012-12-09 13:08:27,859::task::588::TaskManager.Task::(_updateState) Task=`0381f171-af28-4669-bd2e-f07526197d66`::moving from state preparing -> state finished
>Thread-2865::DEBUG::2012-12-09 13:08:27,860::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2865::DEBUG::2012-12-09 13:08:27,860::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2865::DEBUG::2012-12-09 13:08:27,860::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2865::DEBUG::2012-12-09 13:08:27,860::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2865::DEBUG::2012-12-09 13:08:27,860::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2865::DEBUG::2012-12-09 13:08:27,861::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2865::DEBUG::2012-12-09 13:08:27,861::task::978::TaskManager.Task::(_decref) Task=`0381f171-af28-4669-bd2e-f07526197d66`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:28,004::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:28,045::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168209 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:30,052::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:30,092::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172297 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:32,099::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:32,160::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025796 s, 39.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:08:32,498::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:32,499::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:32,603::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:32,604::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:32,604::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:32,605::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2868::DEBUG::2012-12-09 13:08:32,793::task::588::TaskManager.Task::(_updateState) Task=`960563f5-cf4d-4771-bc33-1943f6b3ac84`::moving from state init -> state preparing
>Thread-2868::INFO::2012-12-09 13:08:32,794::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2868::INFO::2012-12-09 13:08:32,794::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124089717865', 'lastCheck': '9.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-2868::DEBUG::2012-12-09 13:08:32,794::task::1172::TaskManager.Task::(prepare) Task=`960563f5-cf4d-4771-bc33-1943f6b3ac84`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124089717865', 'lastCheck': '9.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-2868::DEBUG::2012-12-09 13:08:32,794::task::588::TaskManager.Task::(_updateState) Task=`960563f5-cf4d-4771-bc33-1943f6b3ac84`::moving from state preparing -> state finished
>Thread-2868::DEBUG::2012-12-09 13:08:32,795::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2868::DEBUG::2012-12-09 13:08:32,795::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2868::DEBUG::2012-12-09 13:08:32,795::task::978::TaskManager.Task::(_decref) Task=`960563f5-cf4d-4771-bc33-1943f6b3ac84`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:08:33,333::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:33,341::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000555413 s, 7.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:34,167::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:34,206::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016311 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:36,213::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:36,253::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168856 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2872::DEBUG::2012-12-09 13:08:37,994::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2872::DEBUG::2012-12-09 13:08:37,995::task::588::TaskManager.Task::(_updateState) Task=`2fc9ec9a-db62-482b-91cb-2cccbd34675b`::moving from state init -> state preparing
>Thread-2872::INFO::2012-12-09 13:08:37,996::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2872::INFO::2012-12-09 13:08:37,996::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2872::DEBUG::2012-12-09 13:08:37,996::task::1172::TaskManager.Task::(prepare) Task=`2fc9ec9a-db62-482b-91cb-2cccbd34675b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2872::DEBUG::2012-12-09 13:08:37,996::task::588::TaskManager.Task::(_updateState) Task=`2fc9ec9a-db62-482b-91cb-2cccbd34675b`::moving from state preparing -> state finished
>Thread-2872::DEBUG::2012-12-09 13:08:37,997::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2872::DEBUG::2012-12-09 13:08:37,997::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2872::DEBUG::2012-12-09 13:08:37,997::task::978::TaskManager.Task::(_decref) Task=`2fc9ec9a-db62-482b-91cb-2cccbd34675b`::ref 0 aborting False
>Thread-2873::DEBUG::2012-12-09 13:08:38,009::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2873::DEBUG::2012-12-09 13:08:38,010::task::588::TaskManager.Task::(_updateState) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::moving from state init -> state preparing
>Thread-2873::INFO::2012-12-09 13:08:38,010::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2873::DEBUG::2012-12-09 13:08:38,010::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f79c494-f068-48b8-b012-7723b92c69c3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2873::DEBUG::2012-12-09 13:08:38,011::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2873::DEBUG::2012-12-09 13:08:38,011::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2873::DEBUG::2012-12-09 13:08:38,011::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4f79c494-f068-48b8-b012-7723b92c69c3`::Granted request
>Thread-2873::DEBUG::2012-12-09 13:08:38,011::task::817::TaskManager.Task::(resourceAcquired) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2873::DEBUG::2012-12-09 13:08:38,012::task::978::TaskManager.Task::(_decref) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::ref 1 aborting False
>Thread-2873::DEBUG::2012-12-09 13:08:38,012::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2873::DEBUG::2012-12-09 13:08:38,013::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2873::DEBUG::2012-12-09 13:08:38,115::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2873::WARNING::2012-12-09 13:08:38,117::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2873::DEBUG::2012-12-09 13:08:38,117::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2873::WARNING::2012-12-09 13:08:38,118::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2873::INFO::2012-12-09 13:08:38,118::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2873::DEBUG::2012-12-09 13:08:38,119::task::1172::TaskManager.Task::(prepare) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2873::DEBUG::2012-12-09 13:08:38,119::task::588::TaskManager.Task::(_updateState) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::moving from state preparing -> state finished
>Thread-2873::DEBUG::2012-12-09 13:08:38,119::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2873::DEBUG::2012-12-09 13:08:38,120::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2873::DEBUG::2012-12-09 13:08:38,120::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2873::DEBUG::2012-12-09 13:08:38,120::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2873::DEBUG::2012-12-09 13:08:38,120::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2873::DEBUG::2012-12-09 13:08:38,121::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2873::DEBUG::2012-12-09 13:08:38,121::task::978::TaskManager.Task::(_decref) Task=`77d81398-cf25-463b-b2b1-a5569dbab270`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:38,261::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:38,306::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211346 s, 48.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:40,313::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:40,354::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170133 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:42,360::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:42,402::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165127 s, 62.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:08:42,606::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:42,606::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:42,712::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:42,713::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:42,713::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:42,714::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2876::DEBUG::2012-12-09 13:08:43,180::task::588::TaskManager.Task::(_updateState) Task=`c5aaefb3-b50a-4aea-bef5-907a9d54c189`::moving from state init -> state preparing
>Thread-2876::INFO::2012-12-09 13:08:43,181::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2876::INFO::2012-12-09 13:08:43,181::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00862002372742', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-2876::DEBUG::2012-12-09 13:08:43,181::task::1172::TaskManager.Task::(prepare) Task=`c5aaefb3-b50a-4aea-bef5-907a9d54c189`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00862002372742', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.5', 'code': 358, 'valid': False}}
>Thread-2876::DEBUG::2012-12-09 13:08:43,181::task::588::TaskManager.Task::(_updateState) Task=`c5aaefb3-b50a-4aea-bef5-907a9d54c189`::moving from state preparing -> state finished
>Thread-2876::DEBUG::2012-12-09 13:08:43,181::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2876::DEBUG::2012-12-09 13:08:43,182::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2876::DEBUG::2012-12-09 13:08:43,182::task::978::TaskManager.Task::(_decref) Task=`c5aaefb3-b50a-4aea-bef5-907a9d54c189`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:08:43,344::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:43,352::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00049747 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:44,409::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:44,450::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173686 s, 59.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:46,457::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:46,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173759 s, 58.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:48,505::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:48,551::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163425 s, 62.7 MB/s\n'; <rc> = 0
>Thread-2880::DEBUG::2012-12-09 13:08:48,558::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2880::DEBUG::2012-12-09 13:08:48,559::task::588::TaskManager.Task::(_updateState) Task=`be926923-d4e6-49cc-afb2-398326466013`::moving from state init -> state preparing
>Thread-2880::INFO::2012-12-09 13:08:48,559::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2880::INFO::2012-12-09 13:08:48,559::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2880::DEBUG::2012-12-09 13:08:48,559::task::1172::TaskManager.Task::(prepare) Task=`be926923-d4e6-49cc-afb2-398326466013`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2880::DEBUG::2012-12-09 13:08:48,560::task::588::TaskManager.Task::(_updateState) Task=`be926923-d4e6-49cc-afb2-398326466013`::moving from state preparing -> state finished
>Thread-2880::DEBUG::2012-12-09 13:08:48,560::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2880::DEBUG::2012-12-09 13:08:48,560::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2880::DEBUG::2012-12-09 13:08:48,560::task::978::TaskManager.Task::(_decref) Task=`be926923-d4e6-49cc-afb2-398326466013`::ref 0 aborting False
>Thread-2881::DEBUG::2012-12-09 13:08:48,575::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2881::DEBUG::2012-12-09 13:08:48,575::task::588::TaskManager.Task::(_updateState) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::moving from state init -> state preparing
>Thread-2881::INFO::2012-12-09 13:08:48,576::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2881::DEBUG::2012-12-09 13:08:48,576::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d9cd1d3a-ee53-42b1-98fb-9c80e8749bfe`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2881::DEBUG::2012-12-09 13:08:48,576::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2881::DEBUG::2012-12-09 13:08:48,576::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2881::DEBUG::2012-12-09 13:08:48,577::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d9cd1d3a-ee53-42b1-98fb-9c80e8749bfe`::Granted request
>Thread-2881::DEBUG::2012-12-09 13:08:48,577::task::817::TaskManager.Task::(resourceAcquired) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2881::DEBUG::2012-12-09 13:08:48,577::task::978::TaskManager.Task::(_decref) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::ref 1 aborting False
>Thread-2881::DEBUG::2012-12-09 13:08:48,578::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2881::DEBUG::2012-12-09 13:08:48,578::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2881::DEBUG::2012-12-09 13:08:48,676::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2881::WARNING::2012-12-09 13:08:48,677::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2881::DEBUG::2012-12-09 13:08:48,677::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2881::WARNING::2012-12-09 13:08:48,678::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2881::INFO::2012-12-09 13:08:48,678::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2881::DEBUG::2012-12-09 13:08:48,679::task::1172::TaskManager.Task::(prepare) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2881::DEBUG::2012-12-09 13:08:48,679::task::588::TaskManager.Task::(_updateState) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::moving from state preparing -> state finished
>Thread-2881::DEBUG::2012-12-09 13:08:48,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2881::DEBUG::2012-12-09 13:08:48,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2881::DEBUG::2012-12-09 13:08:48,679::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2881::DEBUG::2012-12-09 13:08:48,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2881::DEBUG::2012-12-09 13:08:48,680::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2881::DEBUG::2012-12-09 13:08:48,680::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2881::DEBUG::2012-12-09 13:08:48,680::task::978::TaskManager.Task::(_decref) Task=`09d806ec-1ac8-49c8-bc82-073ba9b44fdd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:50,560::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:50,601::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166633 s, 61.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:52,607::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:52,646::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164981 s, 62.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:08:52,715::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:08:52,716::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:08:52,814::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:08:52,815::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:08:52,815::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:08:52,816::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:08:53,355::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:08:53,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000493014 s, 8.3 MB/s\n'; <rc> = 0
>Thread-2884::DEBUG::2012-12-09 13:08:53,595::task::588::TaskManager.Task::(_updateState) Task=`0d58a393-5ad1-465e-95cf-961142996160`::moving from state init -> state preparing
>Thread-2884::INFO::2012-12-09 13:08:53,595::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2884::INFO::2012-12-09 13:08:53,595::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120210647583', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-2884::DEBUG::2012-12-09 13:08:53,596::task::1172::TaskManager.Task::(prepare) Task=`0d58a393-5ad1-465e-95cf-961142996160`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120210647583', 'lastCheck': '0.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-2884::DEBUG::2012-12-09 13:08:53,596::task::588::TaskManager.Task::(_updateState) Task=`0d58a393-5ad1-465e-95cf-961142996160`::moving from state preparing -> state finished
>Thread-2884::DEBUG::2012-12-09 13:08:53,596::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2884::DEBUG::2012-12-09 13:08:53,596::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2884::DEBUG::2012-12-09 13:08:53,596::task::978::TaskManager.Task::(_decref) Task=`0d58a393-5ad1-465e-95cf-961142996160`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:08:54,653::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:54,693::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163897 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:56,700::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:56,741::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170876 s, 59.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:08:58,748::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:08:58,788::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162325 s, 63.1 MB/s\n'; <rc> = 0
>Thread-2888::DEBUG::2012-12-09 13:08:58,803::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2888::DEBUG::2012-12-09 13:08:58,803::task::588::TaskManager.Task::(_updateState) Task=`42bd0e2a-0ff9-438b-ba8b-b56a28b52146`::moving from state init -> state preparing
>Thread-2888::INFO::2012-12-09 13:08:58,803::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2888::INFO::2012-12-09 13:08:58,804::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2888::DEBUG::2012-12-09 13:08:58,804::task::1172::TaskManager.Task::(prepare) Task=`42bd0e2a-0ff9-438b-ba8b-b56a28b52146`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2888::DEBUG::2012-12-09 13:08:58,804::task::588::TaskManager.Task::(_updateState) Task=`42bd0e2a-0ff9-438b-ba8b-b56a28b52146`::moving from state preparing -> state finished
>Thread-2888::DEBUG::2012-12-09 13:08:58,804::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2888::DEBUG::2012-12-09 13:08:58,805::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2888::DEBUG::2012-12-09 13:08:58,805::task::978::TaskManager.Task::(_decref) Task=`42bd0e2a-0ff9-438b-ba8b-b56a28b52146`::ref 0 aborting False
>Thread-2889::DEBUG::2012-12-09 13:08:58,821::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2889::DEBUG::2012-12-09 13:08:58,821::task::588::TaskManager.Task::(_updateState) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::moving from state init -> state preparing
>Thread-2889::INFO::2012-12-09 13:08:58,822::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2889::DEBUG::2012-12-09 13:08:58,822::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`74ddfd18-0c4c-4e47-b412-fc9bf544c350`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2889::DEBUG::2012-12-09 13:08:58,823::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2889::DEBUG::2012-12-09 13:08:58,823::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2889::DEBUG::2012-12-09 13:08:58,823::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`74ddfd18-0c4c-4e47-b412-fc9bf544c350`::Granted request
>Thread-2889::DEBUG::2012-12-09 13:08:58,824::task::817::TaskManager.Task::(resourceAcquired) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2889::DEBUG::2012-12-09 13:08:58,824::task::978::TaskManager.Task::(_decref) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::ref 1 aborting False
>Thread-2889::DEBUG::2012-12-09 13:08:58,826::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2889::DEBUG::2012-12-09 13:08:58,826::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2889::DEBUG::2012-12-09 13:08:58,924::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2889::WARNING::2012-12-09 13:08:58,925::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2889::DEBUG::2012-12-09 13:08:58,926::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2889::WARNING::2012-12-09 13:08:58,926::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2889::INFO::2012-12-09 13:08:58,927::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2889::DEBUG::2012-12-09 13:08:58,927::task::1172::TaskManager.Task::(prepare) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2889::DEBUG::2012-12-09 13:08:58,927::task::588::TaskManager.Task::(_updateState) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::moving from state preparing -> state finished
>Thread-2889::DEBUG::2012-12-09 13:08:58,927::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2889::DEBUG::2012-12-09 13:08:58,927::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2889::DEBUG::2012-12-09 13:08:58,928::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2889::DEBUG::2012-12-09 13:08:58,928::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2889::DEBUG::2012-12-09 13:08:58,928::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2889::DEBUG::2012-12-09 13:08:58,928::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2889::DEBUG::2012-12-09 13:08:58,929::task::978::TaskManager.Task::(_decref) Task=`2f69a1db-a8ad-43ee-a71a-4e6f4c065024`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:00,796::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:00,836::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165349 s, 61.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:02,816::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:02,817::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:02,842::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:02,882::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170368 s, 60.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:02,917::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:02,918::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:02,919::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:02,919::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:09:03,369::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:03,394::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000465483 s, 8.8 MB/s\n'; <rc> = 0
>Thread-2892::DEBUG::2012-12-09 13:09:04,039::task::588::TaskManager.Task::(_updateState) Task=`dfdffc08-6a9e-49c6-acd5-b1a5ab7eb1c7`::moving from state init -> state preparing
>Thread-2892::INFO::2012-12-09 13:09:04,039::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2892::INFO::2012-12-09 13:09:04,040::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0252339839935', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2892::DEBUG::2012-12-09 13:09:04,040::task::1172::TaskManager.Task::(prepare) Task=`dfdffc08-6a9e-49c6-acd5-b1a5ab7eb1c7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0252339839935', 'lastCheck': '0.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.1', 'code': 358, 'valid': False}}
>Thread-2892::DEBUG::2012-12-09 13:09:04,040::task::588::TaskManager.Task::(_updateState) Task=`dfdffc08-6a9e-49c6-acd5-b1a5ab7eb1c7`::moving from state preparing -> state finished
>Thread-2892::DEBUG::2012-12-09 13:09:04,040::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2892::DEBUG::2012-12-09 13:09:04,041::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2892::DEBUG::2012-12-09 13:09:04,041::task::978::TaskManager.Task::(_decref) Task=`dfdffc08-6a9e-49c6-acd5-b1a5ab7eb1c7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:04,889::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:04,929::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163372 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:06,937::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:06,978::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173541 s, 59.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:08,985::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:09,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176569 s, 58.0 MB/s\n'; <rc> = 0
>Thread-2896::DEBUG::2012-12-09 13:09:09,102::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2896::DEBUG::2012-12-09 13:09:09,102::task::588::TaskManager.Task::(_updateState) Task=`36d33fef-9f28-454e-b147-4de91cdacc06`::moving from state init -> state preparing
>Thread-2896::INFO::2012-12-09 13:09:09,103::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2896::INFO::2012-12-09 13:09:09,103::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2896::DEBUG::2012-12-09 13:09:09,103::task::1172::TaskManager.Task::(prepare) Task=`36d33fef-9f28-454e-b147-4de91cdacc06`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2896::DEBUG::2012-12-09 13:09:09,103::task::588::TaskManager.Task::(_updateState) Task=`36d33fef-9f28-454e-b147-4de91cdacc06`::moving from state preparing -> state finished
>Thread-2896::DEBUG::2012-12-09 13:09:09,104::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2896::DEBUG::2012-12-09 13:09:09,104::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2896::DEBUG::2012-12-09 13:09:09,104::task::978::TaskManager.Task::(_decref) Task=`36d33fef-9f28-454e-b147-4de91cdacc06`::ref 0 aborting False
>Thread-2897::DEBUG::2012-12-09 13:09:09,117::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2897::DEBUG::2012-12-09 13:09:09,117::task::588::TaskManager.Task::(_updateState) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::moving from state init -> state preparing
>Thread-2897::INFO::2012-12-09 13:09:09,118::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2897::DEBUG::2012-12-09 13:09:09,118::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0b664a9b-f5a1-4224-8519-a577c6dad0df`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2897::DEBUG::2012-12-09 13:09:09,119::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2897::DEBUG::2012-12-09 13:09:09,119::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2897::DEBUG::2012-12-09 13:09:09,119::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0b664a9b-f5a1-4224-8519-a577c6dad0df`::Granted request
>Thread-2897::DEBUG::2012-12-09 13:09:09,120::task::817::TaskManager.Task::(resourceAcquired) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2897::DEBUG::2012-12-09 13:09:09,120::task::978::TaskManager.Task::(_decref) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::ref 1 aborting False
>Thread-2897::DEBUG::2012-12-09 13:09:09,122::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2897::DEBUG::2012-12-09 13:09:09,122::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2897::DEBUG::2012-12-09 13:09:09,246::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2897::WARNING::2012-12-09 13:09:09,247::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2897::DEBUG::2012-12-09 13:09:09,248::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2897::WARNING::2012-12-09 13:09:09,248::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2897::INFO::2012-12-09 13:09:09,249::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2897::DEBUG::2012-12-09 13:09:09,249::task::1172::TaskManager.Task::(prepare) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2897::DEBUG::2012-12-09 13:09:09,250::task::588::TaskManager.Task::(_updateState) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::moving from state preparing -> state finished
>Thread-2897::DEBUG::2012-12-09 13:09:09,250::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2897::DEBUG::2012-12-09 13:09:09,250::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2897::DEBUG::2012-12-09 13:09:09,250::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2897::DEBUG::2012-12-09 13:09:09,251::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2897::DEBUG::2012-12-09 13:09:09,251::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2897::DEBUG::2012-12-09 13:09:09,251::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2897::DEBUG::2012-12-09 13:09:09,251::task::978::TaskManager.Task::(_decref) Task=`5b8c1213-3d00-4a64-bd85-eafa84c4fda4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:11,033::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:11,073::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169904 s, 60.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:12,920::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:12,921::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:09:13,019::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:13,020::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:13,021::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:13,021::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:09:13,080::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:13,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164463 s, 62.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:13,397::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 13:09:13,397::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:09:13,397::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:13,408::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000581081 s, 3.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:13,409::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000581081 s, 3.5 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 13:09:13,409::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 13:09:13,409::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 13:09:13,409::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:09:13,410::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,410::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,410::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,410::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,410::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,411::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:13,508::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:13,509::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:09:13,510::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 13:09:13,510::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 13:09:13,511::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 13:09:13,511::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:09:13,511::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:13,610::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:13,610::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:13,618::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000409359 s, 10.0 MB/s\n'; <rc> = 0
>Thread-2900::DEBUG::2012-12-09 13:09:14,441::task::588::TaskManager.Task::(_updateState) Task=`8c35c36b-cf5d-4d61-8889-62d4f1a71538`::moving from state init -> state preparing
>Thread-2900::INFO::2012-12-09 13:09:14,442::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2900::INFO::2012-12-09 13:09:14,442::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00835585594177', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-2900::DEBUG::2012-12-09 13:09:14,442::task::1172::TaskManager.Task::(prepare) Task=`8c35c36b-cf5d-4d61-8889-62d4f1a71538`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00835585594177', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-2900::DEBUG::2012-12-09 13:09:14,443::task::588::TaskManager.Task::(_updateState) Task=`8c35c36b-cf5d-4d61-8889-62d4f1a71538`::moving from state preparing -> state finished
>Thread-2900::DEBUG::2012-12-09 13:09:14,443::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2900::DEBUG::2012-12-09 13:09:14,443::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2900::DEBUG::2012-12-09 13:09:14,443::task::978::TaskManager.Task::(_decref) Task=`8c35c36b-cf5d-4d61-8889-62d4f1a71538`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:15,126::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:15,167::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185621 s, 55.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:17,174::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:17,224::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0258361 s, 39.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:19,231::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:19,271::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165868 s, 61.7 MB/s\n'; <rc> = 0
>Thread-2904::DEBUG::2012-12-09 13:09:19,600::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2904::DEBUG::2012-12-09 13:09:19,601::task::588::TaskManager.Task::(_updateState) Task=`a234f358-0242-42e2-82ec-1dd4abd24d1a`::moving from state init -> state preparing
>Thread-2904::INFO::2012-12-09 13:09:19,601::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2904::INFO::2012-12-09 13:09:19,601::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2904::DEBUG::2012-12-09 13:09:19,602::task::1172::TaskManager.Task::(prepare) Task=`a234f358-0242-42e2-82ec-1dd4abd24d1a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2904::DEBUG::2012-12-09 13:09:19,602::task::588::TaskManager.Task::(_updateState) Task=`a234f358-0242-42e2-82ec-1dd4abd24d1a`::moving from state preparing -> state finished
>Thread-2904::DEBUG::2012-12-09 13:09:19,602::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2904::DEBUG::2012-12-09 13:09:19,602::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2904::DEBUG::2012-12-09 13:09:19,602::task::978::TaskManager.Task::(_decref) Task=`a234f358-0242-42e2-82ec-1dd4abd24d1a`::ref 0 aborting False
>Thread-2905::DEBUG::2012-12-09 13:09:19,617::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2905::DEBUG::2012-12-09 13:09:19,617::task::588::TaskManager.Task::(_updateState) Task=`082488c3-cd75-450e-8599-88488ef6f171`::moving from state init -> state preparing
>Thread-2905::INFO::2012-12-09 13:09:19,618::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2905::DEBUG::2012-12-09 13:09:19,618::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79c2d116-c10a-41d2-9b2f-9a8874dec0e5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2905::DEBUG::2012-12-09 13:09:19,618::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2905::DEBUG::2012-12-09 13:09:19,619::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2905::DEBUG::2012-12-09 13:09:19,619::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`79c2d116-c10a-41d2-9b2f-9a8874dec0e5`::Granted request
>Thread-2905::DEBUG::2012-12-09 13:09:19,620::task::817::TaskManager.Task::(resourceAcquired) Task=`082488c3-cd75-450e-8599-88488ef6f171`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2905::DEBUG::2012-12-09 13:09:19,620::task::978::TaskManager.Task::(_decref) Task=`082488c3-cd75-450e-8599-88488ef6f171`::ref 1 aborting False
>Thread-2905::DEBUG::2012-12-09 13:09:19,621::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2905::DEBUG::2012-12-09 13:09:19,621::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2905::DEBUG::2012-12-09 13:09:19,719::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2905::WARNING::2012-12-09 13:09:19,720::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2905::DEBUG::2012-12-09 13:09:19,720::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2905::WARNING::2012-12-09 13:09:19,721::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2905::INFO::2012-12-09 13:09:19,721::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2905::DEBUG::2012-12-09 13:09:19,722::task::1172::TaskManager.Task::(prepare) Task=`082488c3-cd75-450e-8599-88488ef6f171`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2905::DEBUG::2012-12-09 13:09:19,722::task::588::TaskManager.Task::(_updateState) Task=`082488c3-cd75-450e-8599-88488ef6f171`::moving from state preparing -> state finished
>Thread-2905::DEBUG::2012-12-09 13:09:19,722::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2905::DEBUG::2012-12-09 13:09:19,722::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2905::DEBUG::2012-12-09 13:09:19,722::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2905::DEBUG::2012-12-09 13:09:19,723::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2905::DEBUG::2012-12-09 13:09:19,723::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2905::DEBUG::2012-12-09 13:09:19,723::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2905::DEBUG::2012-12-09 13:09:19,723::task::978::TaskManager.Task::(_decref) Task=`082488c3-cd75-450e-8599-88488ef6f171`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:21,278::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:21,317::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160743 s, 63.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:23,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:23,023::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:09:23,140::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:23,141::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:23,141::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:23,141::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:09:23,324::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:23,363::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158237 s, 64.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:23,621::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:23,637::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000855137 s, 4.8 MB/s\n'; <rc> = 0
>Thread-2908::DEBUG::2012-12-09 13:09:24,891::task::588::TaskManager.Task::(_updateState) Task=`63daee3d-2f96-4790-822e-4a4e989ad768`::moving from state init -> state preparing
>Thread-2908::INFO::2012-12-09 13:09:24,891::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2908::INFO::2012-12-09 13:09:24,891::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166771411896', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2908::DEBUG::2012-12-09 13:09:24,892::task::1172::TaskManager.Task::(prepare) Task=`63daee3d-2f96-4790-822e-4a4e989ad768`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0166771411896', 'lastCheck': '1.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False}}
>Thread-2908::DEBUG::2012-12-09 13:09:24,892::task::588::TaskManager.Task::(_updateState) Task=`63daee3d-2f96-4790-822e-4a4e989ad768`::moving from state preparing -> state finished
>Thread-2908::DEBUG::2012-12-09 13:09:24,892::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2908::DEBUG::2012-12-09 13:09:24,893::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2908::DEBUG::2012-12-09 13:09:24,893::task::978::TaskManager.Task::(_decref) Task=`63daee3d-2f96-4790-822e-4a4e989ad768`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:25,369::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:25,410::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169522 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:27,417::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:27,464::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241853 s, 42.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:29,471::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:29,511::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169108 s, 60.6 MB/s\n'; <rc> = 0
>Thread-2912::DEBUG::2012-12-09 13:09:29,871::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2912::DEBUG::2012-12-09 13:09:29,872::task::588::TaskManager.Task::(_updateState) Task=`92301b67-1b1c-429e-987b-4ed3d9dc255e`::moving from state init -> state preparing
>Thread-2912::INFO::2012-12-09 13:09:29,872::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2912::INFO::2012-12-09 13:09:29,872::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2912::DEBUG::2012-12-09 13:09:29,872::task::1172::TaskManager.Task::(prepare) Task=`92301b67-1b1c-429e-987b-4ed3d9dc255e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2912::DEBUG::2012-12-09 13:09:29,873::task::588::TaskManager.Task::(_updateState) Task=`92301b67-1b1c-429e-987b-4ed3d9dc255e`::moving from state preparing -> state finished
>Thread-2912::DEBUG::2012-12-09 13:09:29,873::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2912::DEBUG::2012-12-09 13:09:29,873::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2912::DEBUG::2012-12-09 13:09:29,873::task::978::TaskManager.Task::(_decref) Task=`92301b67-1b1c-429e-987b-4ed3d9dc255e`::ref 0 aborting False
>Thread-2913::DEBUG::2012-12-09 13:09:29,886::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2913::DEBUG::2012-12-09 13:09:29,886::task::588::TaskManager.Task::(_updateState) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::moving from state init -> state preparing
>Thread-2913::INFO::2012-12-09 13:09:29,887::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2913::DEBUG::2012-12-09 13:09:29,887::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9b0640c3-d5d7-459a-9328-58dad73c4276`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2913::DEBUG::2012-12-09 13:09:29,887::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2913::DEBUG::2012-12-09 13:09:29,888::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2913::DEBUG::2012-12-09 13:09:29,888::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`9b0640c3-d5d7-459a-9328-58dad73c4276`::Granted request
>Thread-2913::DEBUG::2012-12-09 13:09:29,888::task::817::TaskManager.Task::(resourceAcquired) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2913::DEBUG::2012-12-09 13:09:29,888::task::978::TaskManager.Task::(_decref) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::ref 1 aborting False
>Thread-2913::DEBUG::2012-12-09 13:09:29,889::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2913::DEBUG::2012-12-09 13:09:29,889::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2913::DEBUG::2012-12-09 13:09:29,986::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2913::WARNING::2012-12-09 13:09:29,987::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2913::DEBUG::2012-12-09 13:09:29,987::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2913::WARNING::2012-12-09 13:09:29,988::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2913::INFO::2012-12-09 13:09:29,988::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2913::DEBUG::2012-12-09 13:09:29,988::task::1172::TaskManager.Task::(prepare) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2913::DEBUG::2012-12-09 13:09:29,988::task::588::TaskManager.Task::(_updateState) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::moving from state preparing -> state finished
>Thread-2913::DEBUG::2012-12-09 13:09:29,988::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2913::DEBUG::2012-12-09 13:09:29,989::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2913::DEBUG::2012-12-09 13:09:29,989::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2913::DEBUG::2012-12-09 13:09:29,989::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2913::DEBUG::2012-12-09 13:09:29,989::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2913::DEBUG::2012-12-09 13:09:29,990::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2913::DEBUG::2012-12-09 13:09:29,990::task::978::TaskManager.Task::(_decref) Task=`89ef9d33-c621-48ac-92b6-9c9c53ede0ed`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:31,518::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:31,563::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174434 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:33,142::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:33,143::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:09:33,242::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:33,243::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:33,243::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:33,244::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:09:33,569::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:33,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016564 s, 61.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:09:33,641::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:33,652::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000403391 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:35,624::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:35,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218753 s, 46.8 MB/s\n'; <rc> = 0
>Thread-2916::DEBUG::2012-12-09 13:09:35,679::task::588::TaskManager.Task::(_updateState) Task=`c5d9e3f3-d3b7-42d6-996c-607d47faf314`::moving from state init -> state preparing
>Thread-2916::INFO::2012-12-09 13:09:35,681::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2916::INFO::2012-12-09 13:09:35,684::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0111229419708', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-2916::DEBUG::2012-12-09 13:09:35,686::task::1172::TaskManager.Task::(prepare) Task=`c5d9e3f3-d3b7-42d6-996c-607d47faf314`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0111229419708', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-2916::DEBUG::2012-12-09 13:09:35,687::task::588::TaskManager.Task::(_updateState) Task=`c5d9e3f3-d3b7-42d6-996c-607d47faf314`::moving from state preparing -> state finished
>Thread-2916::DEBUG::2012-12-09 13:09:35,688::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2916::DEBUG::2012-12-09 13:09:35,689::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2916::DEBUG::2012-12-09 13:09:35,690::task::978::TaskManager.Task::(_decref) Task=`c5d9e3f3-d3b7-42d6-996c-607d47faf314`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:37,692::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:37,731::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160351 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:39,738::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:39,780::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185041 s, 55.3 MB/s\n'; <rc> = 0
>Thread-2920::DEBUG::2012-12-09 13:09:40,894::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2920::DEBUG::2012-12-09 13:09:40,894::task::588::TaskManager.Task::(_updateState) Task=`b3077360-5759-4875-9246-f17227b3d891`::moving from state init -> state preparing
>Thread-2920::INFO::2012-12-09 13:09:40,895::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2920::INFO::2012-12-09 13:09:40,895::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2920::DEBUG::2012-12-09 13:09:40,895::task::1172::TaskManager.Task::(prepare) Task=`b3077360-5759-4875-9246-f17227b3d891`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2920::DEBUG::2012-12-09 13:09:40,895::task::588::TaskManager.Task::(_updateState) Task=`b3077360-5759-4875-9246-f17227b3d891`::moving from state preparing -> state finished
>Thread-2920::DEBUG::2012-12-09 13:09:40,895::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2920::DEBUG::2012-12-09 13:09:40,896::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2920::DEBUG::2012-12-09 13:09:40,896::task::978::TaskManager.Task::(_decref) Task=`b3077360-5759-4875-9246-f17227b3d891`::ref 0 aborting False
>Thread-2921::DEBUG::2012-12-09 13:09:40,909::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2921::DEBUG::2012-12-09 13:09:40,910::task::588::TaskManager.Task::(_updateState) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::moving from state init -> state preparing
>Thread-2921::INFO::2012-12-09 13:09:40,910::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2921::DEBUG::2012-12-09 13:09:40,911::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e288c299-839b-4c93-8110-41ec7575da06`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2921::DEBUG::2012-12-09 13:09:40,911::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2921::DEBUG::2012-12-09 13:09:40,911::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2921::DEBUG::2012-12-09 13:09:40,911::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e288c299-839b-4c93-8110-41ec7575da06`::Granted request
>Thread-2921::DEBUG::2012-12-09 13:09:40,912::task::817::TaskManager.Task::(resourceAcquired) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2921::DEBUG::2012-12-09 13:09:40,912::task::978::TaskManager.Task::(_decref) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::ref 1 aborting False
>Thread-2921::DEBUG::2012-12-09 13:09:40,913::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2921::DEBUG::2012-12-09 13:09:40,914::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2921::DEBUG::2012-12-09 13:09:41,019::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2921::WARNING::2012-12-09 13:09:41,020::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2921::DEBUG::2012-12-09 13:09:41,021::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2921::WARNING::2012-12-09 13:09:41,021::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2921::INFO::2012-12-09 13:09:41,022::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2921::DEBUG::2012-12-09 13:09:41,022::task::1172::TaskManager.Task::(prepare) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2921::DEBUG::2012-12-09 13:09:41,022::task::588::TaskManager.Task::(_updateState) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::moving from state preparing -> state finished
>Thread-2921::DEBUG::2012-12-09 13:09:41,023::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2921::DEBUG::2012-12-09 13:09:41,023::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2921::DEBUG::2012-12-09 13:09:41,023::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2921::DEBUG::2012-12-09 13:09:41,023::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2921::DEBUG::2012-12-09 13:09:41,024::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2921::DEBUG::2012-12-09 13:09:41,024::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2921::DEBUG::2012-12-09 13:09:41,024::task::978::TaskManager.Task::(_decref) Task=`509b6274-d3a8-4b25-af73-422826440e0b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:41,787::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:41,838::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168016 s, 60.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:43,244::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:09:43,245::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:43,245::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:09:43,343::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:43,344::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:43,344::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:43,345::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:09:43,655::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:43,665::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000524137 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:43,845::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:43,885::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167043 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:45,892::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:45,931::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160242 s, 63.9 MB/s\n'; <rc> = 0
>Thread-2924::DEBUG::2012-12-09 13:09:46,281::task::588::TaskManager.Task::(_updateState) Task=`32fce045-bb7c-471e-8ea2-550f5c103f50`::moving from state init -> state preparing
>Thread-2924::INFO::2012-12-09 13:09:46,282::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2924::INFO::2012-12-09 13:09:46,282::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0106699466705', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-2924::DEBUG::2012-12-09 13:09:46,282::task::1172::TaskManager.Task::(prepare) Task=`32fce045-bb7c-471e-8ea2-550f5c103f50`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0106699466705', 'lastCheck': '2.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-2924::DEBUG::2012-12-09 13:09:46,282::task::588::TaskManager.Task::(_updateState) Task=`32fce045-bb7c-471e-8ea2-550f5c103f50`::moving from state preparing -> state finished
>Thread-2924::DEBUG::2012-12-09 13:09:46,283::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2924::DEBUG::2012-12-09 13:09:46,283::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2924::DEBUG::2012-12-09 13:09:46,283::task::978::TaskManager.Task::(_decref) Task=`32fce045-bb7c-471e-8ea2-550f5c103f50`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:47,938::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:47,980::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178705 s, 57.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:49,986::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:50,026::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164925 s, 62.1 MB/s\n'; <rc> = 0
>Thread-2928::DEBUG::2012-12-09 13:09:51,177::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2928::DEBUG::2012-12-09 13:09:51,177::task::588::TaskManager.Task::(_updateState) Task=`13281fcb-2667-4ec0-aea7-62655dfe70f9`::moving from state init -> state preparing
>Thread-2928::INFO::2012-12-09 13:09:51,178::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2928::INFO::2012-12-09 13:09:51,178::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2928::DEBUG::2012-12-09 13:09:51,178::task::1172::TaskManager.Task::(prepare) Task=`13281fcb-2667-4ec0-aea7-62655dfe70f9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2928::DEBUG::2012-12-09 13:09:51,178::task::588::TaskManager.Task::(_updateState) Task=`13281fcb-2667-4ec0-aea7-62655dfe70f9`::moving from state preparing -> state finished
>Thread-2928::DEBUG::2012-12-09 13:09:51,178::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2928::DEBUG::2012-12-09 13:09:51,179::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2928::DEBUG::2012-12-09 13:09:51,179::task::978::TaskManager.Task::(_decref) Task=`13281fcb-2667-4ec0-aea7-62655dfe70f9`::ref 0 aborting False
>Thread-2929::DEBUG::2012-12-09 13:09:51,193::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2929::DEBUG::2012-12-09 13:09:51,194::task::588::TaskManager.Task::(_updateState) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::moving from state init -> state preparing
>Thread-2929::INFO::2012-12-09 13:09:51,194::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2929::DEBUG::2012-12-09 13:09:51,194::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`034530f6-96c9-42ad-ace6-6170bb3e34cd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2929::DEBUG::2012-12-09 13:09:51,194::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2929::DEBUG::2012-12-09 13:09:51,195::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2929::DEBUG::2012-12-09 13:09:51,195::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`034530f6-96c9-42ad-ace6-6170bb3e34cd`::Granted request
>Thread-2929::DEBUG::2012-12-09 13:09:51,195::task::817::TaskManager.Task::(resourceAcquired) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2929::DEBUG::2012-12-09 13:09:51,195::task::978::TaskManager.Task::(_decref) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::ref 1 aborting False
>Thread-2929::DEBUG::2012-12-09 13:09:51,196::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2929::DEBUG::2012-12-09 13:09:51,196::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2929::DEBUG::2012-12-09 13:09:51,296::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2929::WARNING::2012-12-09 13:09:51,297::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2929::DEBUG::2012-12-09 13:09:51,298::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2929::WARNING::2012-12-09 13:09:51,298::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2929::INFO::2012-12-09 13:09:51,299::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2929::DEBUG::2012-12-09 13:09:51,299::task::1172::TaskManager.Task::(prepare) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2929::DEBUG::2012-12-09 13:09:51,299::task::588::TaskManager.Task::(_updateState) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::moving from state preparing -> state finished
>Thread-2929::DEBUG::2012-12-09 13:09:51,299::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2929::DEBUG::2012-12-09 13:09:51,299::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2929::DEBUG::2012-12-09 13:09:51,300::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2929::DEBUG::2012-12-09 13:09:51,300::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2929::DEBUG::2012-12-09 13:09:51,300::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2929::DEBUG::2012-12-09 13:09:51,300::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2929::DEBUG::2012-12-09 13:09:51,301::task::978::TaskManager.Task::(_decref) Task=`29de4b3e-78f6-4453-8b5e-6971b5084053`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:52,033::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:52,073::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161694 s, 63.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:09:53,346::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:09:53,347::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:09:53,453::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:09:53,454::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:09:53,454::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:09:53,455::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:09:53,668::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:09:53,680::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00062011 s, 6.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:54,080::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:54,120::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170373 s, 60.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:09:56,127::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:56,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167602 s, 61.1 MB/s\n'; <rc> = 0
>Thread-2932::DEBUG::2012-12-09 13:09:56,515::task::588::TaskManager.Task::(_updateState) Task=`a26c09f2-c41c-4758-bd9b-d756f028b9f7`::moving from state init -> state preparing
>Thread-2932::INFO::2012-12-09 13:09:56,515::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2932::INFO::2012-12-09 13:09:56,516::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126540660858', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2932::DEBUG::2012-12-09 13:09:56,516::task::1172::TaskManager.Task::(prepare) Task=`a26c09f2-c41c-4758-bd9b-d756f028b9f7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0126540660858', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.1', 'code': 358, 'valid': False}}
>Thread-2932::DEBUG::2012-12-09 13:09:56,516::task::588::TaskManager.Task::(_updateState) Task=`a26c09f2-c41c-4758-bd9b-d756f028b9f7`::moving from state preparing -> state finished
>Thread-2932::DEBUG::2012-12-09 13:09:56,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2932::DEBUG::2012-12-09 13:09:56,517::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2932::DEBUG::2012-12-09 13:09:56,517::task::978::TaskManager.Task::(_decref) Task=`a26c09f2-c41c-4758-bd9b-d756f028b9f7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:09:58,175::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:09:58,214::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160504 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:00,221::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:00,271::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248054 s, 41.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:02,278::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:02,318::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181227 s, 56.5 MB/s\n'; <rc> = 0
>Thread-2936::DEBUG::2012-12-09 13:10:02,358::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2936::DEBUG::2012-12-09 13:10:02,359::task::588::TaskManager.Task::(_updateState) Task=`011b2734-d4af-4939-9bae-6cd679a38ef8`::moving from state init -> state preparing
>Thread-2936::INFO::2012-12-09 13:10:02,360::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2936::INFO::2012-12-09 13:10:02,360::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2936::DEBUG::2012-12-09 13:10:02,360::task::1172::TaskManager.Task::(prepare) Task=`011b2734-d4af-4939-9bae-6cd679a38ef8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2936::DEBUG::2012-12-09 13:10:02,361::task::588::TaskManager.Task::(_updateState) Task=`011b2734-d4af-4939-9bae-6cd679a38ef8`::moving from state preparing -> state finished
>Thread-2936::DEBUG::2012-12-09 13:10:02,361::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2936::DEBUG::2012-12-09 13:10:02,361::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2936::DEBUG::2012-12-09 13:10:02,361::task::978::TaskManager.Task::(_decref) Task=`011b2734-d4af-4939-9bae-6cd679a38ef8`::ref 0 aborting False
>Thread-2937::DEBUG::2012-12-09 13:10:02,374::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2937::DEBUG::2012-12-09 13:10:02,375::task::588::TaskManager.Task::(_updateState) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::moving from state init -> state preparing
>Thread-2937::INFO::2012-12-09 13:10:02,375::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2937::DEBUG::2012-12-09 13:10:02,375::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`757150b6-31bb-43a4-b672-1b6f261f1d5e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2937::DEBUG::2012-12-09 13:10:02,375::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2937::DEBUG::2012-12-09 13:10:02,376::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2937::DEBUG::2012-12-09 13:10:02,376::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`757150b6-31bb-43a4-b672-1b6f261f1d5e`::Granted request
>Thread-2937::DEBUG::2012-12-09 13:10:02,376::task::817::TaskManager.Task::(resourceAcquired) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2937::DEBUG::2012-12-09 13:10:02,376::task::978::TaskManager.Task::(_decref) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::ref 1 aborting False
>Thread-2937::DEBUG::2012-12-09 13:10:02,377::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2937::DEBUG::2012-12-09 13:10:02,378::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2937::DEBUG::2012-12-09 13:10:02,477::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2937::WARNING::2012-12-09 13:10:02,478::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2937::DEBUG::2012-12-09 13:10:02,479::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2937::WARNING::2012-12-09 13:10:02,479::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2937::INFO::2012-12-09 13:10:02,480::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2937::DEBUG::2012-12-09 13:10:02,480::task::1172::TaskManager.Task::(prepare) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2937::DEBUG::2012-12-09 13:10:02,480::task::588::TaskManager.Task::(_updateState) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::moving from state preparing -> state finished
>Thread-2937::DEBUG::2012-12-09 13:10:02,480::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2937::DEBUG::2012-12-09 13:10:02,480::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2937::DEBUG::2012-12-09 13:10:02,481::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2937::DEBUG::2012-12-09 13:10:02,481::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2937::DEBUG::2012-12-09 13:10:02,481::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2937::DEBUG::2012-12-09 13:10:02,481::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2937::DEBUG::2012-12-09 13:10:02,481::task::978::TaskManager.Task::(_decref) Task=`09fb6806-ebbc-4ebb-8912-57789b8ea11c`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:10:03,456::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:03,457::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:10:03,558::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:03,559::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:03,559::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:03,559::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:10:03,684::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:03,695::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000447951 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:04,325::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:04,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015805 s, 64.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:06,372::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:06,412::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166569 s, 61.5 MB/s\n'; <rc> = 0
>Thread-2940::DEBUG::2012-12-09 13:10:06,937::task::588::TaskManager.Task::(_updateState) Task=`c00852d2-44df-415d-8df3-5c51aa0f1d98`::moving from state init -> state preparing
>Thread-2940::INFO::2012-12-09 13:10:06,937::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2940::INFO::2012-12-09 13:10:06,938::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118341445923', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-2940::DEBUG::2012-12-09 13:10:06,938::task::1172::TaskManager.Task::(prepare) Task=`c00852d2-44df-415d-8df3-5c51aa0f1d98`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118341445923', 'lastCheck': '3.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-2940::DEBUG::2012-12-09 13:10:06,938::task::588::TaskManager.Task::(_updateState) Task=`c00852d2-44df-415d-8df3-5c51aa0f1d98`::moving from state preparing -> state finished
>Thread-2940::DEBUG::2012-12-09 13:10:06,938::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2940::DEBUG::2012-12-09 13:10:06,939::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2940::DEBUG::2012-12-09 13:10:06,939::task::978::TaskManager.Task::(_decref) Task=`c00852d2-44df-415d-8df3-5c51aa0f1d98`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:08,419::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:08,461::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159041 s, 64.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:10,468::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:10,509::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164815 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:12,516::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:12,557::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163646 s, 62.6 MB/s\n'; <rc> = 0
>Thread-2944::DEBUG::2012-12-09 13:10:13,297::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2944::DEBUG::2012-12-09 13:10:13,298::task::588::TaskManager.Task::(_updateState) Task=`cfd026e6-4924-43bd-8451-8a3fc87684dc`::moving from state init -> state preparing
>Thread-2944::INFO::2012-12-09 13:10:13,298::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2944::INFO::2012-12-09 13:10:13,298::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2944::DEBUG::2012-12-09 13:10:13,298::task::1172::TaskManager.Task::(prepare) Task=`cfd026e6-4924-43bd-8451-8a3fc87684dc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2944::DEBUG::2012-12-09 13:10:13,299::task::588::TaskManager.Task::(_updateState) Task=`cfd026e6-4924-43bd-8451-8a3fc87684dc`::moving from state preparing -> state finished
>Thread-2944::DEBUG::2012-12-09 13:10:13,299::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2944::DEBUG::2012-12-09 13:10:13,299::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2944::DEBUG::2012-12-09 13:10:13,299::task::978::TaskManager.Task::(_decref) Task=`cfd026e6-4924-43bd-8451-8a3fc87684dc`::ref 0 aborting False
>Thread-2945::DEBUG::2012-12-09 13:10:13,312::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2945::DEBUG::2012-12-09 13:10:13,312::task::588::TaskManager.Task::(_updateState) Task=`ff847639-839b-4d49-a45a-e586279609ec`::moving from state init -> state preparing
>Thread-2945::INFO::2012-12-09 13:10:13,313::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2945::DEBUG::2012-12-09 13:10:13,313::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8617e93a-cec4-4f5a-859a-462e86485711`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2945::DEBUG::2012-12-09 13:10:13,313::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2945::DEBUG::2012-12-09 13:10:13,313::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2945::DEBUG::2012-12-09 13:10:13,314::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8617e93a-cec4-4f5a-859a-462e86485711`::Granted request
>Thread-2945::DEBUG::2012-12-09 13:10:13,314::task::817::TaskManager.Task::(resourceAcquired) Task=`ff847639-839b-4d49-a45a-e586279609ec`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2945::DEBUG::2012-12-09 13:10:13,314::task::978::TaskManager.Task::(_decref) Task=`ff847639-839b-4d49-a45a-e586279609ec`::ref 1 aborting False
>Thread-2945::DEBUG::2012-12-09 13:10:13,315::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2945::DEBUG::2012-12-09 13:10:13,315::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2945::DEBUG::2012-12-09 13:10:13,412::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2945::WARNING::2012-12-09 13:10:13,413::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2945::DEBUG::2012-12-09 13:10:13,413::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2945::WARNING::2012-12-09 13:10:13,414::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2945::INFO::2012-12-09 13:10:13,414::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2945::DEBUG::2012-12-09 13:10:13,414::task::1172::TaskManager.Task::(prepare) Task=`ff847639-839b-4d49-a45a-e586279609ec`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2945::DEBUG::2012-12-09 13:10:13,414::task::588::TaskManager.Task::(_updateState) Task=`ff847639-839b-4d49-a45a-e586279609ec`::moving from state preparing -> state finished
>Thread-2945::DEBUG::2012-12-09 13:10:13,415::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2945::DEBUG::2012-12-09 13:10:13,415::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2945::DEBUG::2012-12-09 13:10:13,415::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2945::DEBUG::2012-12-09 13:10:13,415::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2945::DEBUG::2012-12-09 13:10:13,416::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2945::DEBUG::2012-12-09 13:10:13,416::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2945::DEBUG::2012-12-09 13:10:13,416::task::978::TaskManager.Task::(_decref) Task=`ff847639-839b-4d49-a45a-e586279609ec`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:10:13,560::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:13,561::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:10:13,659::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:13,660::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:13,660::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:13,660::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:10:13,698::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:13,706::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000498887 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:14,564::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:14,602::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154247 s, 66.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:16,609::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:16,649::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162909 s, 62.9 MB/s\n'; <rc> = 0
>Thread-2948::DEBUG::2012-12-09 13:10:17,835::task::588::TaskManager.Task::(_updateState) Task=`b6057da8-8c74-44c6-a148-790b6b634ce0`::moving from state init -> state preparing
>Thread-2948::INFO::2012-12-09 13:10:17,836::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2948::INFO::2012-12-09 13:10:17,837::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00856304168701', 'lastCheck': '4.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2948::DEBUG::2012-12-09 13:10:17,837::task::1172::TaskManager.Task::(prepare) Task=`b6057da8-8c74-44c6-a148-790b6b634ce0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00856304168701', 'lastCheck': '4.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-2948::DEBUG::2012-12-09 13:10:17,837::task::588::TaskManager.Task::(_updateState) Task=`b6057da8-8c74-44c6-a148-790b6b634ce0`::moving from state preparing -> state finished
>Thread-2948::DEBUG::2012-12-09 13:10:17,838::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2948::DEBUG::2012-12-09 13:10:17,838::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2948::DEBUG::2012-12-09 13:10:17,838::task::978::TaskManager.Task::(_decref) Task=`b6057da8-8c74-44c6-a148-790b6b634ce0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:18,656::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:18,699::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171445 s, 59.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:20,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:20,748::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168989 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:22,754::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:22,795::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170108 s, 60.2 MB/s\n'; <rc> = 0
>Thread-2952::DEBUG::2012-12-09 13:10:23,483::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2952::DEBUG::2012-12-09 13:10:23,483::task::588::TaskManager.Task::(_updateState) Task=`c4ce6ec7-4057-4f1b-9910-3674b0693b92`::moving from state init -> state preparing
>Thread-2952::INFO::2012-12-09 13:10:23,483::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2952::INFO::2012-12-09 13:10:23,484::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2952::DEBUG::2012-12-09 13:10:23,484::task::1172::TaskManager.Task::(prepare) Task=`c4ce6ec7-4057-4f1b-9910-3674b0693b92`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2952::DEBUG::2012-12-09 13:10:23,484::task::588::TaskManager.Task::(_updateState) Task=`c4ce6ec7-4057-4f1b-9910-3674b0693b92`::moving from state preparing -> state finished
>Thread-2952::DEBUG::2012-12-09 13:10:23,484::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2952::DEBUG::2012-12-09 13:10:23,485::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2952::DEBUG::2012-12-09 13:10:23,485::task::978::TaskManager.Task::(_decref) Task=`c4ce6ec7-4057-4f1b-9910-3674b0693b92`::ref 0 aborting False
>Thread-2953::DEBUG::2012-12-09 13:10:23,501::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2953::DEBUG::2012-12-09 13:10:23,501::task::588::TaskManager.Task::(_updateState) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::moving from state init -> state preparing
>Thread-2953::INFO::2012-12-09 13:10:23,501::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2953::DEBUG::2012-12-09 13:10:23,502::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8e915e38-2031-4667-a66c-910ca82d7d5f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2953::DEBUG::2012-12-09 13:10:23,502::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2953::DEBUG::2012-12-09 13:10:23,502::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2953::DEBUG::2012-12-09 13:10:23,502::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8e915e38-2031-4667-a66c-910ca82d7d5f`::Granted request
>Thread-2953::DEBUG::2012-12-09 13:10:23,503::task::817::TaskManager.Task::(resourceAcquired) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2953::DEBUG::2012-12-09 13:10:23,503::task::978::TaskManager.Task::(_decref) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::ref 1 aborting False
>Thread-2953::DEBUG::2012-12-09 13:10:23,504::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2953::DEBUG::2012-12-09 13:10:23,504::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2953::DEBUG::2012-12-09 13:10:23,602::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2953::WARNING::2012-12-09 13:10:23,603::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2953::DEBUG::2012-12-09 13:10:23,604::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2953::WARNING::2012-12-09 13:10:23,604::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2953::INFO::2012-12-09 13:10:23,605::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2953::DEBUG::2012-12-09 13:10:23,605::task::1172::TaskManager.Task::(prepare) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2953::DEBUG::2012-12-09 13:10:23,605::task::588::TaskManager.Task::(_updateState) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::moving from state preparing -> state finished
>Thread-2953::DEBUG::2012-12-09 13:10:23,605::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2953::DEBUG::2012-12-09 13:10:23,606::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2953::DEBUG::2012-12-09 13:10:23,606::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2953::DEBUG::2012-12-09 13:10:23,606::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2953::DEBUG::2012-12-09 13:10:23,606::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2953::DEBUG::2012-12-09 13:10:23,606::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2953::DEBUG::2012-12-09 13:10:23,607::task::978::TaskManager.Task::(_decref) Task=`a92e0b22-d89f-4722-9e49-83866e878971`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:10:23,661::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:23,662::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:23,709::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:23,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000411953 s, 9.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:10:23,763::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:23,764::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:23,764::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:23,764::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:10:24,802::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:24,844::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171734 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:26,851::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:26,890::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164782 s, 62.1 MB/s\n'; <rc> = 0
>Thread-2956::DEBUG::2012-12-09 13:10:28,706::task::588::TaskManager.Task::(_updateState) Task=`b4f7c9c1-b2e3-44bb-97e3-acc8ed532b1d`::moving from state init -> state preparing
>Thread-2956::INFO::2012-12-09 13:10:28,707::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2956::INFO::2012-12-09 13:10:28,707::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0255398750305', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2956::DEBUG::2012-12-09 13:10:28,707::task::1172::TaskManager.Task::(prepare) Task=`b4f7c9c1-b2e3-44bb-97e3-acc8ed532b1d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0255398750305', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-2956::DEBUG::2012-12-09 13:10:28,707::task::588::TaskManager.Task::(_updateState) Task=`b4f7c9c1-b2e3-44bb-97e3-acc8ed532b1d`::moving from state preparing -> state finished
>Thread-2956::DEBUG::2012-12-09 13:10:28,708::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2956::DEBUG::2012-12-09 13:10:28,708::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2956::DEBUG::2012-12-09 13:10:28,708::task::978::TaskManager.Task::(_decref) Task=`b4f7c9c1-b2e3-44bb-97e3-acc8ed532b1d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:28,897::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:28,938::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172243 s, 59.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:30,944::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:30,985::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174581 s, 58.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:32,992::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:33,032::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175132 s, 58.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:10:33,738::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:33,746::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0004328 s, 9.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:10:33,765::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:33,766::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:10:33,871::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:33,872::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:33,873::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:33,873::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2960::DEBUG::2012-12-09 13:10:34,584::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2960::DEBUG::2012-12-09 13:10:34,584::task::588::TaskManager.Task::(_updateState) Task=`765c5ff3-77c0-47ba-ab13-127a1f2d27d5`::moving from state init -> state preparing
>Thread-2960::INFO::2012-12-09 13:10:34,585::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2960::INFO::2012-12-09 13:10:34,585::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2960::DEBUG::2012-12-09 13:10:34,585::task::1172::TaskManager.Task::(prepare) Task=`765c5ff3-77c0-47ba-ab13-127a1f2d27d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2960::DEBUG::2012-12-09 13:10:34,585::task::588::TaskManager.Task::(_updateState) Task=`765c5ff3-77c0-47ba-ab13-127a1f2d27d5`::moving from state preparing -> state finished
>Thread-2960::DEBUG::2012-12-09 13:10:34,585::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2960::DEBUG::2012-12-09 13:10:34,586::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2960::DEBUG::2012-12-09 13:10:34,586::task::978::TaskManager.Task::(_decref) Task=`765c5ff3-77c0-47ba-ab13-127a1f2d27d5`::ref 0 aborting False
>Thread-2961::DEBUG::2012-12-09 13:10:34,598::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2961::DEBUG::2012-12-09 13:10:34,598::task::588::TaskManager.Task::(_updateState) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::moving from state init -> state preparing
>Thread-2961::INFO::2012-12-09 13:10:34,599::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2961::DEBUG::2012-12-09 13:10:34,599::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`56f631a3-0059-4c64-aa2b-a2b8a08ed99c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2961::DEBUG::2012-12-09 13:10:34,599::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2961::DEBUG::2012-12-09 13:10:34,599::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2961::DEBUG::2012-12-09 13:10:34,600::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`56f631a3-0059-4c64-aa2b-a2b8a08ed99c`::Granted request
>Thread-2961::DEBUG::2012-12-09 13:10:34,600::task::817::TaskManager.Task::(resourceAcquired) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2961::DEBUG::2012-12-09 13:10:34,600::task::978::TaskManager.Task::(_decref) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::ref 1 aborting False
>Thread-2961::DEBUG::2012-12-09 13:10:34,601::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2961::DEBUG::2012-12-09 13:10:34,601::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2961::DEBUG::2012-12-09 13:10:34,699::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2961::WARNING::2012-12-09 13:10:34,700::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2961::DEBUG::2012-12-09 13:10:34,700::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2961::WARNING::2012-12-09 13:10:34,701::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2961::INFO::2012-12-09 13:10:34,701::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2961::DEBUG::2012-12-09 13:10:34,701::task::1172::TaskManager.Task::(prepare) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2961::DEBUG::2012-12-09 13:10:34,701::task::588::TaskManager.Task::(_updateState) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::moving from state preparing -> state finished
>Thread-2961::DEBUG::2012-12-09 13:10:34,702::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2961::DEBUG::2012-12-09 13:10:34,702::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2961::DEBUG::2012-12-09 13:10:34,702::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2961::DEBUG::2012-12-09 13:10:34,702::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2961::DEBUG::2012-12-09 13:10:34,702::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2961::DEBUG::2012-12-09 13:10:34,703::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2961::DEBUG::2012-12-09 13:10:34,703::task::978::TaskManager.Task::(_decref) Task=`6bec2efd-548a-4f86-964a-b5abf0e5325c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:35,040::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:35,079::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166173 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:37,086::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:37,127::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169066 s, 60.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:39,133::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:39,174::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175324 s, 58.4 MB/s\n'; <rc> = 0
>Thread-2964::DEBUG::2012-12-09 13:10:39,560::task::588::TaskManager.Task::(_updateState) Task=`fee3154a-9c42-4012-934e-7e8d24f73aa2`::moving from state init -> state preparing
>Thread-2964::INFO::2012-12-09 13:10:39,561::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2964::INFO::2012-12-09 13:10:39,561::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00863194465637', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-2964::DEBUG::2012-12-09 13:10:39,561::task::1172::TaskManager.Task::(prepare) Task=`fee3154a-9c42-4012-934e-7e8d24f73aa2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00863194465637', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-2964::DEBUG::2012-12-09 13:10:39,561::task::588::TaskManager.Task::(_updateState) Task=`fee3154a-9c42-4012-934e-7e8d24f73aa2`::moving from state preparing -> state finished
>Thread-2964::DEBUG::2012-12-09 13:10:39,562::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2964::DEBUG::2012-12-09 13:10:39,562::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2964::DEBUG::2012-12-09 13:10:39,562::task::978::TaskManager.Task::(_decref) Task=`fee3154a-9c42-4012-934e-7e8d24f73aa2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:41,181::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:41,221::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172866 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:43,228::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:43,269::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165973 s, 61.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:10:43,749::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:43,758::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000484369 s, 8.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:10:43,874::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:43,875::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:10:43,975::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:43,977::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:43,977::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:43,978::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2968::DEBUG::2012-12-09 13:10:45,198::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2968::DEBUG::2012-12-09 13:10:45,199::task::588::TaskManager.Task::(_updateState) Task=`d4a2ae30-faa4-48eb-87eb-aa0e71036402`::moving from state init -> state preparing
>Thread-2968::INFO::2012-12-09 13:10:45,199::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2968::INFO::2012-12-09 13:10:45,200::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2968::DEBUG::2012-12-09 13:10:45,200::task::1172::TaskManager.Task::(prepare) Task=`d4a2ae30-faa4-48eb-87eb-aa0e71036402`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2968::DEBUG::2012-12-09 13:10:45,201::task::588::TaskManager.Task::(_updateState) Task=`d4a2ae30-faa4-48eb-87eb-aa0e71036402`::moving from state preparing -> state finished
>Thread-2968::DEBUG::2012-12-09 13:10:45,201::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2968::DEBUG::2012-12-09 13:10:45,202::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2968::DEBUG::2012-12-09 13:10:45,202::task::978::TaskManager.Task::(_decref) Task=`d4a2ae30-faa4-48eb-87eb-aa0e71036402`::ref 0 aborting False
>Thread-2969::DEBUG::2012-12-09 13:10:45,215::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2969::DEBUG::2012-12-09 13:10:45,216::task::588::TaskManager.Task::(_updateState) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::moving from state init -> state preparing
>Thread-2969::INFO::2012-12-09 13:10:45,216::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2969::DEBUG::2012-12-09 13:10:45,217::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`936460d6-8589-40c3-9f7b-a5a92c7ea8bc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2969::DEBUG::2012-12-09 13:10:45,217::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2969::DEBUG::2012-12-09 13:10:45,217::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2969::DEBUG::2012-12-09 13:10:45,217::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`936460d6-8589-40c3-9f7b-a5a92c7ea8bc`::Granted request
>Thread-2969::DEBUG::2012-12-09 13:10:45,217::task::817::TaskManager.Task::(resourceAcquired) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2969::DEBUG::2012-12-09 13:10:45,218::task::978::TaskManager.Task::(_decref) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::ref 1 aborting False
>Thread-2969::DEBUG::2012-12-09 13:10:45,218::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2969::DEBUG::2012-12-09 13:10:45,219::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:45,275::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:45,317::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181952 s, 56.3 MB/s\n'; <rc> = 0
>Thread-2969::DEBUG::2012-12-09 13:10:45,321::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2969::WARNING::2012-12-09 13:10:45,331::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2969::DEBUG::2012-12-09 13:10:45,331::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2969::WARNING::2012-12-09 13:10:45,331::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2969::INFO::2012-12-09 13:10:45,332::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2969::DEBUG::2012-12-09 13:10:45,332::task::1172::TaskManager.Task::(prepare) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2969::DEBUG::2012-12-09 13:10:45,332::task::588::TaskManager.Task::(_updateState) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::moving from state preparing -> state finished
>Thread-2969::DEBUG::2012-12-09 13:10:45,332::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2969::DEBUG::2012-12-09 13:10:45,333::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2969::DEBUG::2012-12-09 13:10:45,333::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2969::DEBUG::2012-12-09 13:10:45,333::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2969::DEBUG::2012-12-09 13:10:45,333::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2969::DEBUG::2012-12-09 13:10:45,333::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2969::DEBUG::2012-12-09 13:10:45,334::task::978::TaskManager.Task::(_decref) Task=`e1eb20c3-d8ba-44f9-b552-208ce13fb732`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:47,333::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:47,380::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206971 s, 49.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:49,387::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:49,436::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239634 s, 42.7 MB/s\n'; <rc> = 0
>Thread-2972::DEBUG::2012-12-09 13:10:50,064::task::588::TaskManager.Task::(_updateState) Task=`da26c328-9c29-4824-8246-a993507429dd`::moving from state init -> state preparing
>Thread-2972::INFO::2012-12-09 13:10:50,065::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2972::INFO::2012-12-09 13:10:50,065::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091290473938', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-2972::DEBUG::2012-12-09 13:10:50,065::task::1172::TaskManager.Task::(prepare) Task=`da26c328-9c29-4824-8246-a993507429dd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091290473938', 'lastCheck': '6.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.1', 'code': 358, 'valid': False}}
>Thread-2972::DEBUG::2012-12-09 13:10:50,065::task::588::TaskManager.Task::(_updateState) Task=`da26c328-9c29-4824-8246-a993507429dd`::moving from state preparing -> state finished
>Thread-2972::DEBUG::2012-12-09 13:10:50,066::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2972::DEBUG::2012-12-09 13:10:50,066::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2972::DEBUG::2012-12-09 13:10:50,066::task::978::TaskManager.Task::(_decref) Task=`da26c328-9c29-4824-8246-a993507429dd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:51,443::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:51,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170513 s, 60.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:53,508::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:53,566::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0209121 s, 49.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:10:53,761::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:10:53,774::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000602058 s, 6.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:10:53,978::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:10:53,979::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:10:54,080::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:10:54,081::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:10:54,081::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:10:54,082::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:10:55,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:55,617::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176626 s, 58.0 MB/s\n'; <rc> = 0
>Thread-2976::DEBUG::2012-12-09 13:10:56,825::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2976::DEBUG::2012-12-09 13:10:56,827::task::588::TaskManager.Task::(_updateState) Task=`50b0f4e6-ae48-496e-a7f9-b5f638c9091e`::moving from state init -> state preparing
>Thread-2976::INFO::2012-12-09 13:10:56,827::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2976::INFO::2012-12-09 13:10:56,827::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2976::DEBUG::2012-12-09 13:10:56,828::task::1172::TaskManager.Task::(prepare) Task=`50b0f4e6-ae48-496e-a7f9-b5f638c9091e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2976::DEBUG::2012-12-09 13:10:56,828::task::588::TaskManager.Task::(_updateState) Task=`50b0f4e6-ae48-496e-a7f9-b5f638c9091e`::moving from state preparing -> state finished
>Thread-2976::DEBUG::2012-12-09 13:10:56,828::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2976::DEBUG::2012-12-09 13:10:56,828::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2976::DEBUG::2012-12-09 13:10:56,828::task::978::TaskManager.Task::(_decref) Task=`50b0f4e6-ae48-496e-a7f9-b5f638c9091e`::ref 0 aborting False
>Thread-2978::DEBUG::2012-12-09 13:10:56,840::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2978::DEBUG::2012-12-09 13:10:56,840::task::588::TaskManager.Task::(_updateState) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::moving from state init -> state preparing
>Thread-2978::INFO::2012-12-09 13:10:56,841::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2978::DEBUG::2012-12-09 13:10:56,841::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd4243f4-4ea5-416e-a2a2-3425f5943b77`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2978::DEBUG::2012-12-09 13:10:56,841::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2978::DEBUG::2012-12-09 13:10:56,841::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2978::DEBUG::2012-12-09 13:10:56,842::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fd4243f4-4ea5-416e-a2a2-3425f5943b77`::Granted request
>Thread-2978::DEBUG::2012-12-09 13:10:56,842::task::817::TaskManager.Task::(resourceAcquired) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2978::DEBUG::2012-12-09 13:10:56,842::task::978::TaskManager.Task::(_decref) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::ref 1 aborting False
>Thread-2978::DEBUG::2012-12-09 13:10:56,843::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2978::DEBUG::2012-12-09 13:10:56,843::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2978::DEBUG::2012-12-09 13:10:56,936::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2978::WARNING::2012-12-09 13:10:56,937::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2978::DEBUG::2012-12-09 13:10:56,937::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2978::WARNING::2012-12-09 13:10:56,938::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2978::INFO::2012-12-09 13:10:56,938::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2978::DEBUG::2012-12-09 13:10:56,939::task::1172::TaskManager.Task::(prepare) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2978::DEBUG::2012-12-09 13:10:56,939::task::588::TaskManager.Task::(_updateState) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::moving from state preparing -> state finished
>Thread-2978::DEBUG::2012-12-09 13:10:56,939::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2978::DEBUG::2012-12-09 13:10:56,939::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2978::DEBUG::2012-12-09 13:10:56,939::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2978::DEBUG::2012-12-09 13:10:56,940::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2978::DEBUG::2012-12-09 13:10:56,940::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2978::DEBUG::2012-12-09 13:10:56,940::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2978::DEBUG::2012-12-09 13:10:56,940::task::978::TaskManager.Task::(_decref) Task=`c3b47bc5-49c4-42de-826b-fb46d3a627b9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:10:57,624::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:57,663::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161937 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:10:59,669::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:10:59,721::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191517 s, 53.5 MB/s\n'; <rc> = 0
>Thread-2980::DEBUG::2012-12-09 13:11:00,870::task::588::TaskManager.Task::(_updateState) Task=`3b2563da-92ae-4d6f-ac6c-57f9d4816036`::moving from state init -> state preparing
>Thread-2980::INFO::2012-12-09 13:11:00,870::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2980::INFO::2012-12-09 13:11:00,871::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136108398438', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.8', 'code': 358, 'valid': False}}
>Thread-2980::DEBUG::2012-12-09 13:11:00,871::task::1172::TaskManager.Task::(prepare) Task=`3b2563da-92ae-4d6f-ac6c-57f9d4816036`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136108398438', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.8', 'code': 358, 'valid': False}}
>Thread-2980::DEBUG::2012-12-09 13:11:00,871::task::588::TaskManager.Task::(_updateState) Task=`3b2563da-92ae-4d6f-ac6c-57f9d4816036`::moving from state preparing -> state finished
>Thread-2980::DEBUG::2012-12-09 13:11:00,871::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2980::DEBUG::2012-12-09 13:11:00,872::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2980::DEBUG::2012-12-09 13:11:00,872::task::978::TaskManager.Task::(_decref) Task=`3b2563da-92ae-4d6f-ac6c-57f9d4816036`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:01,727::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:01,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163996 s, 62.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:03,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:03,783::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:03,797::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00038803 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:03,837::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0402735 s, 25.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:04,083::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:04,083::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:11:04,180::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:04,181::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:04,182::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:04,182::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:11:05,844::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:05,884::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168821 s, 60.7 MB/s\n'; <rc> = 0
>Thread-2984::DEBUG::2012-12-09 13:11:07,844::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2984::DEBUG::2012-12-09 13:11:07,845::task::588::TaskManager.Task::(_updateState) Task=`bcd83a80-8341-46f1-a069-e1f25cd24996`::moving from state init -> state preparing
>Thread-2984::INFO::2012-12-09 13:11:07,845::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2984::INFO::2012-12-09 13:11:07,846::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2984::DEBUG::2012-12-09 13:11:07,846::task::1172::TaskManager.Task::(prepare) Task=`bcd83a80-8341-46f1-a069-e1f25cd24996`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2984::DEBUG::2012-12-09 13:11:07,846::task::588::TaskManager.Task::(_updateState) Task=`bcd83a80-8341-46f1-a069-e1f25cd24996`::moving from state preparing -> state finished
>Thread-2984::DEBUG::2012-12-09 13:11:07,847::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2984::DEBUG::2012-12-09 13:11:07,847::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2984::DEBUG::2012-12-09 13:11:07,847::task::978::TaskManager.Task::(_decref) Task=`bcd83a80-8341-46f1-a069-e1f25cd24996`::ref 0 aborting False
>Thread-2985::DEBUG::2012-12-09 13:11:07,862::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2985::DEBUG::2012-12-09 13:11:07,862::task::588::TaskManager.Task::(_updateState) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::moving from state init -> state preparing
>Thread-2985::INFO::2012-12-09 13:11:07,863::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2985::DEBUG::2012-12-09 13:11:07,863::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3587eabe-e468-4acf-89ba-8470a087d902`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2985::DEBUG::2012-12-09 13:11:07,864::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2985::DEBUG::2012-12-09 13:11:07,864::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2985::DEBUG::2012-12-09 13:11:07,865::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3587eabe-e468-4acf-89ba-8470a087d902`::Granted request
>Thread-2985::DEBUG::2012-12-09 13:11:07,865::task::817::TaskManager.Task::(resourceAcquired) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2985::DEBUG::2012-12-09 13:11:07,865::task::978::TaskManager.Task::(_decref) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::ref 1 aborting False
>Thread-2985::DEBUG::2012-12-09 13:11:07,867::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2985::DEBUG::2012-12-09 13:11:07,868::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:07,891::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:07,954::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025226 s, 40.6 MB/s\n'; <rc> = 0
>Thread-2985::DEBUG::2012-12-09 13:11:07,976::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2985::WARNING::2012-12-09 13:11:07,977::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2985::DEBUG::2012-12-09 13:11:07,977::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2985::WARNING::2012-12-09 13:11:07,977::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2985::INFO::2012-12-09 13:11:07,978::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2985::DEBUG::2012-12-09 13:11:07,978::task::1172::TaskManager.Task::(prepare) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2985::DEBUG::2012-12-09 13:11:07,978::task::588::TaskManager.Task::(_updateState) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::moving from state preparing -> state finished
>Thread-2985::DEBUG::2012-12-09 13:11:07,978::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2985::DEBUG::2012-12-09 13:11:07,979::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2985::DEBUG::2012-12-09 13:11:07,979::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2985::DEBUG::2012-12-09 13:11:07,979::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2985::DEBUG::2012-12-09 13:11:07,979::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2985::DEBUG::2012-12-09 13:11:07,979::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2985::DEBUG::2012-12-09 13:11:07,980::task::978::TaskManager.Task::(_decref) Task=`c89734ad-c3b8-4299-b7fb-5a6ce323cd0c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:09,961::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:10,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161787 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:12,007::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:12,055::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223274 s, 45.9 MB/s\n'; <rc> = 0
>Thread-2988::DEBUG::2012-12-09 13:11:12,392::task::588::TaskManager.Task::(_updateState) Task=`4f04f468-39e2-4e26-9b33-5b2b10fe8763`::moving from state init -> state preparing
>Thread-2988::INFO::2012-12-09 13:11:12,393::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2988::INFO::2012-12-09 13:11:12,393::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152840614319', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-2988::DEBUG::2012-12-09 13:11:12,394::task::1172::TaskManager.Task::(prepare) Task=`4f04f468-39e2-4e26-9b33-5b2b10fe8763`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0152840614319', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-2988::DEBUG::2012-12-09 13:11:12,394::task::588::TaskManager.Task::(_updateState) Task=`4f04f468-39e2-4e26-9b33-5b2b10fe8763`::moving from state preparing -> state finished
>Thread-2988::DEBUG::2012-12-09 13:11:12,394::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2988::DEBUG::2012-12-09 13:11:12,394::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2988::DEBUG::2012-12-09 13:11:12,394::task::978::TaskManager.Task::(_decref) Task=`4f04f468-39e2-4e26-9b33-5b2b10fe8763`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:11:13,804::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:13,816::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000431009 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:14,061::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:14,102::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017123 s, 59.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:14,183::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:14,184::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:11:14,286::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:14,287::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:14,287::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:14,288::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:11:16,109::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:16,151::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165822 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:18,157::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:18,198::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164599 s, 62.2 MB/s\n'; <rc> = 0
>Thread-2992::DEBUG::2012-12-09 13:11:19,008::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2992::DEBUG::2012-12-09 13:11:19,008::task::588::TaskManager.Task::(_updateState) Task=`e883997f-3a7a-410b-9a35-4e83c876890e`::moving from state init -> state preparing
>Thread-2992::INFO::2012-12-09 13:11:19,008::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2992::INFO::2012-12-09 13:11:19,009::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2992::DEBUG::2012-12-09 13:11:19,009::task::1172::TaskManager.Task::(prepare) Task=`e883997f-3a7a-410b-9a35-4e83c876890e`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-2992::DEBUG::2012-12-09 13:11:19,009::task::588::TaskManager.Task::(_updateState) Task=`e883997f-3a7a-410b-9a35-4e83c876890e`::moving from state preparing -> state finished
>Thread-2992::DEBUG::2012-12-09 13:11:19,009::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2992::DEBUG::2012-12-09 13:11:19,009::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2992::DEBUG::2012-12-09 13:11:19,010::task::978::TaskManager.Task::(_decref) Task=`e883997f-3a7a-410b-9a35-4e83c876890e`::ref 0 aborting False
>Thread-2993::DEBUG::2012-12-09 13:11:19,022::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-2993::DEBUG::2012-12-09 13:11:19,023::task::588::TaskManager.Task::(_updateState) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::moving from state init -> state preparing
>Thread-2993::INFO::2012-12-09 13:11:19,023::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-2993::DEBUG::2012-12-09 13:11:19,024::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d8d03edb-2a8f-4098-86ec-0dcb00ba939a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-2993::DEBUG::2012-12-09 13:11:19,024::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-2993::DEBUG::2012-12-09 13:11:19,025::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-2993::DEBUG::2012-12-09 13:11:19,025::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d8d03edb-2a8f-4098-86ec-0dcb00ba939a`::Granted request
>Thread-2993::DEBUG::2012-12-09 13:11:19,026::task::817::TaskManager.Task::(resourceAcquired) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-2993::DEBUG::2012-12-09 13:11:19,026::task::978::TaskManager.Task::(_decref) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::ref 1 aborting False
>Thread-2993::DEBUG::2012-12-09 13:11:19,028::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-2993::DEBUG::2012-12-09 13:11:19,029::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-2993::DEBUG::2012-12-09 13:11:19,136::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-2993::WARNING::2012-12-09 13:11:19,137::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-2993::DEBUG::2012-12-09 13:11:19,137::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-2993::WARNING::2012-12-09 13:11:19,138::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-2993::INFO::2012-12-09 13:11:19,138::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2993::DEBUG::2012-12-09 13:11:19,139::task::1172::TaskManager.Task::(prepare) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-2993::DEBUG::2012-12-09 13:11:19,139::task::588::TaskManager.Task::(_updateState) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::moving from state preparing -> state finished
>Thread-2993::DEBUG::2012-12-09 13:11:19,139::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-2993::DEBUG::2012-12-09 13:11:19,140::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2993::DEBUG::2012-12-09 13:11:19,140::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-2993::DEBUG::2012-12-09 13:11:19,140::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-2993::DEBUG::2012-12-09 13:11:19,140::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-2993::DEBUG::2012-12-09 13:11:19,141::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-2993::DEBUG::2012-12-09 13:11:19,141::task::978::TaskManager.Task::(_decref) Task=`c0923ac0-ba28-4fbb-8a61-abf5cc48c731`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:20,205::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:20,255::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172825 s, 59.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:22,260::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:22,302::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180798 s, 56.6 MB/s\n'; <rc> = 0
>Thread-2996::DEBUG::2012-12-09 13:11:23,499::task::588::TaskManager.Task::(_updateState) Task=`ba59a0ad-789b-4758-9867-f2b02c60c044`::moving from state init -> state preparing
>Thread-2996::INFO::2012-12-09 13:11:23,500::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2996::INFO::2012-12-09 13:11:23,500::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125091075897', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-2996::DEBUG::2012-12-09 13:11:23,501::task::1172::TaskManager.Task::(prepare) Task=`ba59a0ad-789b-4758-9867-f2b02c60c044`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0125091075897', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.2', 'code': 358, 'valid': False}}
>Thread-2996::DEBUG::2012-12-09 13:11:23,501::task::588::TaskManager.Task::(_updateState) Task=`ba59a0ad-789b-4758-9867-f2b02c60c044`::moving from state preparing -> state finished
>Thread-2996::DEBUG::2012-12-09 13:11:23,501::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-2996::DEBUG::2012-12-09 13:11:23,502::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-2996::DEBUG::2012-12-09 13:11:23,502::task::978::TaskManager.Task::(_decref) Task=`ba59a0ad-789b-4758-9867-f2b02c60c044`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:11:23,819::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:23,831::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000372678 s, 11.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:24,288::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:24,289::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:24,308::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:24,355::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190499 s, 53.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:24,396::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:24,397::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:24,398::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:24,398::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:11:26,362::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:26,402::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186078 s, 55.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:28,409::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:28,451::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176545 s, 58.0 MB/s\n'; <rc> = 0
>Thread-3001::DEBUG::2012-12-09 13:11:30,311::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3001::DEBUG::2012-12-09 13:11:30,312::task::588::TaskManager.Task::(_updateState) Task=`268d3b7f-2b43-4fee-9004-5dc2cace984d`::moving from state init -> state preparing
>Thread-3001::INFO::2012-12-09 13:11:30,312::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3001::INFO::2012-12-09 13:11:30,313::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3001::DEBUG::2012-12-09 13:11:30,313::task::1172::TaskManager.Task::(prepare) Task=`268d3b7f-2b43-4fee-9004-5dc2cace984d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3001::DEBUG::2012-12-09 13:11:30,314::task::588::TaskManager.Task::(_updateState) Task=`268d3b7f-2b43-4fee-9004-5dc2cace984d`::moving from state preparing -> state finished
>Thread-3001::DEBUG::2012-12-09 13:11:30,314::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3001::DEBUG::2012-12-09 13:11:30,314::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3001::DEBUG::2012-12-09 13:11:30,315::task::978::TaskManager.Task::(_decref) Task=`268d3b7f-2b43-4fee-9004-5dc2cace984d`::ref 0 aborting False
>Thread-3002::DEBUG::2012-12-09 13:11:30,336::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3002::DEBUG::2012-12-09 13:11:30,336::task::588::TaskManager.Task::(_updateState) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::moving from state init -> state preparing
>Thread-3002::INFO::2012-12-09 13:11:30,337::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3002::DEBUG::2012-12-09 13:11:30,337::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3f41a26a-464e-451e-b789-8f0aa3670834`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3002::DEBUG::2012-12-09 13:11:30,337::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3002::DEBUG::2012-12-09 13:11:30,337::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3002::DEBUG::2012-12-09 13:11:30,338::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3f41a26a-464e-451e-b789-8f0aa3670834`::Granted request
>Thread-3002::DEBUG::2012-12-09 13:11:30,338::task::817::TaskManager.Task::(resourceAcquired) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3002::DEBUG::2012-12-09 13:11:30,338::task::978::TaskManager.Task::(_decref) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::ref 1 aborting False
>Thread-3002::DEBUG::2012-12-09 13:11:30,339::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3002::DEBUG::2012-12-09 13:11:30,339::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3002::DEBUG::2012-12-09 13:11:30,438::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3002::WARNING::2012-12-09 13:11:30,439::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3002::DEBUG::2012-12-09 13:11:30,440::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3002::WARNING::2012-12-09 13:11:30,440::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3002::INFO::2012-12-09 13:11:30,440::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3002::DEBUG::2012-12-09 13:11:30,441::task::1172::TaskManager.Task::(prepare) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3002::DEBUG::2012-12-09 13:11:30,441::task::588::TaskManager.Task::(_updateState) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::moving from state preparing -> state finished
>Thread-3002::DEBUG::2012-12-09 13:11:30,441::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3002::DEBUG::2012-12-09 13:11:30,441::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3002::DEBUG::2012-12-09 13:11:30,441::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3002::DEBUG::2012-12-09 13:11:30,442::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3002::DEBUG::2012-12-09 13:11:30,442::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3002::DEBUG::2012-12-09 13:11:30,442::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3002::DEBUG::2012-12-09 13:11:30,442::task::978::TaskManager.Task::(_decref) Task=`f8ef7a08-4bd5-49f6-89a4-228f9c299d83`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:30,458::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:30,501::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195019 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:32,508::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:32,549::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176249 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:11:33,834::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:33,842::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000482291 s, 8.5 MB/s\n'; <rc> = 0
>Thread-3004::DEBUG::2012-12-09 13:11:34,272::task::588::TaskManager.Task::(_updateState) Task=`bf40ebda-5443-4ddd-a135-a4e7cc6f37ad`::moving from state init -> state preparing
>Thread-3004::INFO::2012-12-09 13:11:34,272::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3004::INFO::2012-12-09 13:11:34,273::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00838303565979', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-3004::DEBUG::2012-12-09 13:11:34,273::task::1172::TaskManager.Task::(prepare) Task=`bf40ebda-5443-4ddd-a135-a4e7cc6f37ad`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00838303565979', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.9', 'code': 358, 'valid': False}}
>Thread-3004::DEBUG::2012-12-09 13:11:34,273::task::588::TaskManager.Task::(_updateState) Task=`bf40ebda-5443-4ddd-a135-a4e7cc6f37ad`::moving from state preparing -> state finished
>Thread-3004::DEBUG::2012-12-09 13:11:34,274::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3004::DEBUG::2012-12-09 13:11:34,274::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3004::DEBUG::2012-12-09 13:11:34,274::task::978::TaskManager.Task::(_decref) Task=`bf40ebda-5443-4ddd-a135-a4e7cc6f37ad`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:11:34,399::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:34,400::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:11:34,499::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:34,500::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:34,500::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:34,501::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:11:34,556::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:34,596::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016622 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:36,602::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:36,646::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169576 s, 60.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:38,652::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:38,696::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185088 s, 55.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:40,704::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:40,747::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172486 s, 59.4 MB/s\n'; <rc> = 0
>Thread-3009::DEBUG::2012-12-09 13:11:41,529::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3009::DEBUG::2012-12-09 13:11:41,530::task::588::TaskManager.Task::(_updateState) Task=`a6a7f25c-93e0-4d31-bccd-83531131ae8a`::moving from state init -> state preparing
>Thread-3009::INFO::2012-12-09 13:11:41,531::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3009::INFO::2012-12-09 13:11:41,531::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3009::DEBUG::2012-12-09 13:11:41,531::task::1172::TaskManager.Task::(prepare) Task=`a6a7f25c-93e0-4d31-bccd-83531131ae8a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3009::DEBUG::2012-12-09 13:11:41,532::task::588::TaskManager.Task::(_updateState) Task=`a6a7f25c-93e0-4d31-bccd-83531131ae8a`::moving from state preparing -> state finished
>Thread-3009::DEBUG::2012-12-09 13:11:41,532::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3009::DEBUG::2012-12-09 13:11:41,532::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3009::DEBUG::2012-12-09 13:11:41,532::task::978::TaskManager.Task::(_decref) Task=`a6a7f25c-93e0-4d31-bccd-83531131ae8a`::ref 0 aborting False
>Thread-3010::DEBUG::2012-12-09 13:11:41,548::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3010::DEBUG::2012-12-09 13:11:41,548::task::588::TaskManager.Task::(_updateState) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::moving from state init -> state preparing
>Thread-3010::INFO::2012-12-09 13:11:41,549::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3010::DEBUG::2012-12-09 13:11:41,549::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f6c84d23-d964-4dc7-981c-7f8536865f35`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3010::DEBUG::2012-12-09 13:11:41,549::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3010::DEBUG::2012-12-09 13:11:41,550::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3010::DEBUG::2012-12-09 13:11:41,550::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f6c84d23-d964-4dc7-981c-7f8536865f35`::Granted request
>Thread-3010::DEBUG::2012-12-09 13:11:41,550::task::817::TaskManager.Task::(resourceAcquired) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3010::DEBUG::2012-12-09 13:11:41,550::task::978::TaskManager.Task::(_decref) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::ref 1 aborting False
>Thread-3010::DEBUG::2012-12-09 13:11:41,551::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3010::DEBUG::2012-12-09 13:11:41,552::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3010::DEBUG::2012-12-09 13:11:41,653::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3010::WARNING::2012-12-09 13:11:41,654::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3010::DEBUG::2012-12-09 13:11:41,655::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3010::WARNING::2012-12-09 13:11:41,655::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3010::INFO::2012-12-09 13:11:41,656::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3010::DEBUG::2012-12-09 13:11:41,656::task::1172::TaskManager.Task::(prepare) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3010::DEBUG::2012-12-09 13:11:41,656::task::588::TaskManager.Task::(_updateState) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::moving from state preparing -> state finished
>Thread-3010::DEBUG::2012-12-09 13:11:41,656::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3010::DEBUG::2012-12-09 13:11:41,656::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3010::DEBUG::2012-12-09 13:11:41,657::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3010::DEBUG::2012-12-09 13:11:41,657::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3010::DEBUG::2012-12-09 13:11:41,657::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3010::DEBUG::2012-12-09 13:11:41,657::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3010::DEBUG::2012-12-09 13:11:41,658::task::978::TaskManager.Task::(_decref) Task=`1e0567bb-f7a2-493c-929c-87d2a1986482`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:42,754::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:42,806::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210362 s, 48.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:11:43,852::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:43,860::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000380925 s, 10.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:44,502::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:44,502::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:11:44,609::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:44,610::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:44,611::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:44,611::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3012::DEBUG::2012-12-09 13:11:44,644::task::588::TaskManager.Task::(_updateState) Task=`3864523e-4921-4a48-af08-133e2e966192`::moving from state init -> state preparing
>Thread-3012::INFO::2012-12-09 13:11:44,645::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3012::INFO::2012-12-09 13:11:44,645::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00899386405945', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-3012::DEBUG::2012-12-09 13:11:44,646::task::1172::TaskManager.Task::(prepare) Task=`3864523e-4921-4a48-af08-133e2e966192`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00899386405945', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.0', 'code': 358, 'valid': False}}
>Thread-3012::DEBUG::2012-12-09 13:11:44,646::task::588::TaskManager.Task::(_updateState) Task=`3864523e-4921-4a48-af08-133e2e966192`::moving from state preparing -> state finished
>Thread-3012::DEBUG::2012-12-09 13:11:44,646::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3012::DEBUG::2012-12-09 13:11:44,646::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3012::DEBUG::2012-12-09 13:11:44,647::task::978::TaskManager.Task::(_decref) Task=`3864523e-4921-4a48-af08-133e2e966192`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:44,813::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:44,852::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165572 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:46,859::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:46,905::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215836 s, 47.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:48,912::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:48,953::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166123 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:50,959::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:51,004::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203304 s, 50.4 MB/s\n'; <rc> = 0
>Thread-3017::DEBUG::2012-12-09 13:11:52,219::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3017::DEBUG::2012-12-09 13:11:52,219::task::588::TaskManager.Task::(_updateState) Task=`8698b248-20fe-4a92-9717-7c7037266e98`::moving from state init -> state preparing
>Thread-3017::INFO::2012-12-09 13:11:52,220::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3017::INFO::2012-12-09 13:11:52,220::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3017::DEBUG::2012-12-09 13:11:52,220::task::1172::TaskManager.Task::(prepare) Task=`8698b248-20fe-4a92-9717-7c7037266e98`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3017::DEBUG::2012-12-09 13:11:52,220::task::588::TaskManager.Task::(_updateState) Task=`8698b248-20fe-4a92-9717-7c7037266e98`::moving from state preparing -> state finished
>Thread-3017::DEBUG::2012-12-09 13:11:52,220::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3017::DEBUG::2012-12-09 13:11:52,221::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3017::DEBUG::2012-12-09 13:11:52,221::task::978::TaskManager.Task::(_decref) Task=`8698b248-20fe-4a92-9717-7c7037266e98`::ref 0 aborting False
>Thread-3018::DEBUG::2012-12-09 13:11:52,238::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3018::DEBUG::2012-12-09 13:11:52,238::task::588::TaskManager.Task::(_updateState) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::moving from state init -> state preparing
>Thread-3018::INFO::2012-12-09 13:11:52,239::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3018::DEBUG::2012-12-09 13:11:52,239::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ded5e6e2-3f97-4f34-b32e-265f791b2f0a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3018::DEBUG::2012-12-09 13:11:52,239::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3018::DEBUG::2012-12-09 13:11:52,240::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3018::DEBUG::2012-12-09 13:11:52,240::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ded5e6e2-3f97-4f34-b32e-265f791b2f0a`::Granted request
>Thread-3018::DEBUG::2012-12-09 13:11:52,240::task::817::TaskManager.Task::(resourceAcquired) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3018::DEBUG::2012-12-09 13:11:52,240::task::978::TaskManager.Task::(_decref) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::ref 1 aborting False
>Thread-3018::DEBUG::2012-12-09 13:11:52,241::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3018::DEBUG::2012-12-09 13:11:52,241::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3018::DEBUG::2012-12-09 13:11:52,346::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3018::WARNING::2012-12-09 13:11:52,347::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3018::DEBUG::2012-12-09 13:11:52,347::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3018::WARNING::2012-12-09 13:11:52,348::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3018::INFO::2012-12-09 13:11:52,348::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3018::DEBUG::2012-12-09 13:11:52,348::task::1172::TaskManager.Task::(prepare) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3018::DEBUG::2012-12-09 13:11:52,348::task::588::TaskManager.Task::(_updateState) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::moving from state preparing -> state finished
>Thread-3018::DEBUG::2012-12-09 13:11:52,349::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3018::DEBUG::2012-12-09 13:11:52,349::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3018::DEBUG::2012-12-09 13:11:52,349::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3018::DEBUG::2012-12-09 13:11:52,349::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3018::DEBUG::2012-12-09 13:11:52,350::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3018::DEBUG::2012-12-09 13:11:52,350::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3018::DEBUG::2012-12-09 13:11:52,350::task::978::TaskManager.Task::(_decref) Task=`a38e9b6a-3e24-4864-b761-cadd8f005eb6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:53,011::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:53,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166898 s, 61.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:11:53,863::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:11:53,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000457992 s, 8.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:11:54,612::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:11:54,613::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:11:54,723::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:11:54,725::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:11:54,725::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:11:54,725::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3020::DEBUG::2012-12-09 13:11:54,949::task::588::TaskManager.Task::(_updateState) Task=`5f09629a-4955-4883-9ac3-3dc5b8edb161`::moving from state init -> state preparing
>Thread-3020::INFO::2012-12-09 13:11:54,950::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3020::INFO::2012-12-09 13:11:54,950::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114159584045', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-3020::DEBUG::2012-12-09 13:11:54,950::task::1172::TaskManager.Task::(prepare) Task=`5f09629a-4955-4883-9ac3-3dc5b8edb161`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114159584045', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.2', 'code': 358, 'valid': False}}
>Thread-3020::DEBUG::2012-12-09 13:11:54,951::task::588::TaskManager.Task::(_updateState) Task=`5f09629a-4955-4883-9ac3-3dc5b8edb161`::moving from state preparing -> state finished
>Thread-3020::DEBUG::2012-12-09 13:11:54,951::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3020::DEBUG::2012-12-09 13:11:54,951::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3020::DEBUG::2012-12-09 13:11:54,951::task::978::TaskManager.Task::(_decref) Task=`5f09629a-4955-4883-9ac3-3dc5b8edb161`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:11:55,057::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:55,121::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254504 s, 40.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:57,131::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:57,171::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175365 s, 58.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:11:59,178::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:11:59,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168203 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:01,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:01,273::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182763 s, 56.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:03,280::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:03,322::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177039 s, 57.8 MB/s\n'; <rc> = 0
>Thread-3025::DEBUG::2012-12-09 13:12:03,338::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3025::DEBUG::2012-12-09 13:12:03,339::task::588::TaskManager.Task::(_updateState) Task=`8a2dad9f-654f-4a14-8a76-b3389e636683`::moving from state init -> state preparing
>Thread-3025::INFO::2012-12-09 13:12:03,339::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3025::INFO::2012-12-09 13:12:03,339::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3025::DEBUG::2012-12-09 13:12:03,340::task::1172::TaskManager.Task::(prepare) Task=`8a2dad9f-654f-4a14-8a76-b3389e636683`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3025::DEBUG::2012-12-09 13:12:03,340::task::588::TaskManager.Task::(_updateState) Task=`8a2dad9f-654f-4a14-8a76-b3389e636683`::moving from state preparing -> state finished
>Thread-3025::DEBUG::2012-12-09 13:12:03,340::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3025::DEBUG::2012-12-09 13:12:03,340::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3025::DEBUG::2012-12-09 13:12:03,340::task::978::TaskManager.Task::(_decref) Task=`8a2dad9f-654f-4a14-8a76-b3389e636683`::ref 0 aborting False
>Thread-3026::DEBUG::2012-12-09 13:12:03,361::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3026::DEBUG::2012-12-09 13:12:03,362::task::588::TaskManager.Task::(_updateState) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::moving from state init -> state preparing
>Thread-3026::INFO::2012-12-09 13:12:03,362::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3026::DEBUG::2012-12-09 13:12:03,363::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a4473e4-e8b9-43ca-a9cd-93febce0f09d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3026::DEBUG::2012-12-09 13:12:03,363::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3026::DEBUG::2012-12-09 13:12:03,363::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3026::DEBUG::2012-12-09 13:12:03,363::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8a4473e4-e8b9-43ca-a9cd-93febce0f09d`::Granted request
>Thread-3026::DEBUG::2012-12-09 13:12:03,364::task::817::TaskManager.Task::(resourceAcquired) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3026::DEBUG::2012-12-09 13:12:03,364::task::978::TaskManager.Task::(_decref) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::ref 1 aborting False
>Thread-3026::DEBUG::2012-12-09 13:12:03,365::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3026::DEBUG::2012-12-09 13:12:03,365::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3026::DEBUG::2012-12-09 13:12:03,478::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3026::WARNING::2012-12-09 13:12:03,479::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3026::DEBUG::2012-12-09 13:12:03,480::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3026::WARNING::2012-12-09 13:12:03,480::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3026::INFO::2012-12-09 13:12:03,481::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3026::DEBUG::2012-12-09 13:12:03,481::task::1172::TaskManager.Task::(prepare) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3026::DEBUG::2012-12-09 13:12:03,481::task::588::TaskManager.Task::(_updateState) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::moving from state preparing -> state finished
>Thread-3026::DEBUG::2012-12-09 13:12:03,481::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3026::DEBUG::2012-12-09 13:12:03,482::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3026::DEBUG::2012-12-09 13:12:03,482::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3026::DEBUG::2012-12-09 13:12:03,482::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3026::DEBUG::2012-12-09 13:12:03,482::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3026::DEBUG::2012-12-09 13:12:03,483::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3026::DEBUG::2012-12-09 13:12:03,483::task::978::TaskManager.Task::(_decref) Task=`0a67ce6c-6332-4d0c-90d4-e199896a32a4`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:12:03,877::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:03,889::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000377942 s, 10.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:12:04,726::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:04,727::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:04,826::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:04,827::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:04,827::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:04,828::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:12:05,331::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:05,370::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165284 s, 62.0 MB/s\n'; <rc> = 0
>Thread-3028::DEBUG::2012-12-09 13:12:05,591::task::588::TaskManager.Task::(_updateState) Task=`ae565a0f-f2a6-41e9-a57c-431c2f671e02`::moving from state init -> state preparing
>Thread-3028::INFO::2012-12-09 13:12:05,592::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3028::INFO::2012-12-09 13:12:05,592::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124278068542', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-3028::DEBUG::2012-12-09 13:12:05,592::task::1172::TaskManager.Task::(prepare) Task=`ae565a0f-f2a6-41e9-a57c-431c2f671e02`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0124278068542', 'lastCheck': '1.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.8', 'code': 358, 'valid': False}}
>Thread-3028::DEBUG::2012-12-09 13:12:05,592::task::588::TaskManager.Task::(_updateState) Task=`ae565a0f-f2a6-41e9-a57c-431c2f671e02`::moving from state preparing -> state finished
>Thread-3028::DEBUG::2012-12-09 13:12:05,592::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3028::DEBUG::2012-12-09 13:12:05,593::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3028::DEBUG::2012-12-09 13:12:05,593::task::978::TaskManager.Task::(_decref) Task=`ae565a0f-f2a6-41e9-a57c-431c2f671e02`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:07,377::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:07,425::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222663 s, 46.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:09,432::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:09,475::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189592 s, 54.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:11,482::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:11,526::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203454 s, 50.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:13,533::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:13,573::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162056 s, 63.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:12:13,892::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:13,906::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000671319 s, 6.1 MB/s\n'; <rc> = 0
>Thread-3033::DEBUG::2012-12-09 13:12:13,927::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3033::DEBUG::2012-12-09 13:12:13,928::task::588::TaskManager.Task::(_updateState) Task=`cf25f46a-0c88-424e-866e-df431e2a339d`::moving from state init -> state preparing
>Thread-3033::INFO::2012-12-09 13:12:13,929::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3033::INFO::2012-12-09 13:12:13,929::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3033::DEBUG::2012-12-09 13:12:13,930::task::1172::TaskManager.Task::(prepare) Task=`cf25f46a-0c88-424e-866e-df431e2a339d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3033::DEBUG::2012-12-09 13:12:13,930::task::588::TaskManager.Task::(_updateState) Task=`cf25f46a-0c88-424e-866e-df431e2a339d`::moving from state preparing -> state finished
>Thread-3033::DEBUG::2012-12-09 13:12:13,930::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3033::DEBUG::2012-12-09 13:12:13,931::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3033::DEBUG::2012-12-09 13:12:13,931::task::978::TaskManager.Task::(_decref) Task=`cf25f46a-0c88-424e-866e-df431e2a339d`::ref 0 aborting False
>Thread-3034::DEBUG::2012-12-09 13:12:13,946::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3034::DEBUG::2012-12-09 13:12:13,946::task::588::TaskManager.Task::(_updateState) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::moving from state init -> state preparing
>Thread-3034::INFO::2012-12-09 13:12:13,947::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3034::DEBUG::2012-12-09 13:12:13,947::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ee2e7945-e527-46b4-82c7-c3b2c9af6cdb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3034::DEBUG::2012-12-09 13:12:13,947::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3034::DEBUG::2012-12-09 13:12:13,947::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3034::DEBUG::2012-12-09 13:12:13,948::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ee2e7945-e527-46b4-82c7-c3b2c9af6cdb`::Granted request
>Thread-3034::DEBUG::2012-12-09 13:12:13,948::task::817::TaskManager.Task::(resourceAcquired) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3034::DEBUG::2012-12-09 13:12:13,948::task::978::TaskManager.Task::(_decref) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::ref 1 aborting False
>Thread-3034::DEBUG::2012-12-09 13:12:13,949::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3034::DEBUG::2012-12-09 13:12:13,949::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3034::DEBUG::2012-12-09 13:12:14,050::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3034::WARNING::2012-12-09 13:12:14,051::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3034::DEBUG::2012-12-09 13:12:14,051::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3034::WARNING::2012-12-09 13:12:14,052::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3034::INFO::2012-12-09 13:12:14,052::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3034::DEBUG::2012-12-09 13:12:14,052::task::1172::TaskManager.Task::(prepare) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3034::DEBUG::2012-12-09 13:12:14,052::task::588::TaskManager.Task::(_updateState) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::moving from state preparing -> state finished
>Thread-3034::DEBUG::2012-12-09 13:12:14,053::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3034::DEBUG::2012-12-09 13:12:14,053::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3034::DEBUG::2012-12-09 13:12:14,053::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3034::DEBUG::2012-12-09 13:12:14,053::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3034::DEBUG::2012-12-09 13:12:14,054::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3034::DEBUG::2012-12-09 13:12:14,054::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3034::DEBUG::2012-12-09 13:12:14,054::task::978::TaskManager.Task::(_decref) Task=`a55bc4c1-52ed-4888-8d83-2f63d2c9517a`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:12:14,828::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:14,829::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:14,932::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:14,933::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:14,934::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:14,935::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:12:15,580::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:15,620::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162447 s, 63.0 MB/s\n'; <rc> = 0
>Thread-3036::DEBUG::2012-12-09 13:12:16,754::task::588::TaskManager.Task::(_updateState) Task=`24826e7c-b2c4-4f4b-b23b-034a27303cec`::moving from state init -> state preparing
>Thread-3036::INFO::2012-12-09 13:12:16,755::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3036::INFO::2012-12-09 13:12:16,755::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146398544312', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-3036::DEBUG::2012-12-09 13:12:16,755::task::1172::TaskManager.Task::(prepare) Task=`24826e7c-b2c4-4f4b-b23b-034a27303cec`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0146398544312', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.8', 'code': 358, 'valid': False}}
>Thread-3036::DEBUG::2012-12-09 13:12:16,756::task::588::TaskManager.Task::(_updateState) Task=`24826e7c-b2c4-4f4b-b23b-034a27303cec`::moving from state preparing -> state finished
>Thread-3036::DEBUG::2012-12-09 13:12:16,756::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3036::DEBUG::2012-12-09 13:12:16,756::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3036::DEBUG::2012-12-09 13:12:16,757::task::978::TaskManager.Task::(_decref) Task=`24826e7c-b2c4-4f4b-b23b-034a27303cec`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:17,627::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:17,668::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170441 s, 60.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:19,675::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:19,715::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017583 s, 58.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:21,723::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:21,765::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183053 s, 55.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:23,772::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:23,818::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172355 s, 59.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:12:23,913::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:23,925::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000392075 s, 10.4 MB/s\n'; <rc> = 0
>Thread-3041::DEBUG::2012-12-09 13:12:24,534::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3041::DEBUG::2012-12-09 13:12:24,535::task::588::TaskManager.Task::(_updateState) Task=`b71e8dc3-8ab0-4940-89e0-850c06394f74`::moving from state init -> state preparing
>Thread-3041::INFO::2012-12-09 13:12:24,535::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3041::INFO::2012-12-09 13:12:24,535::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3041::DEBUG::2012-12-09 13:12:24,536::task::1172::TaskManager.Task::(prepare) Task=`b71e8dc3-8ab0-4940-89e0-850c06394f74`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3041::DEBUG::2012-12-09 13:12:24,536::task::588::TaskManager.Task::(_updateState) Task=`b71e8dc3-8ab0-4940-89e0-850c06394f74`::moving from state preparing -> state finished
>Thread-3041::DEBUG::2012-12-09 13:12:24,536::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3041::DEBUG::2012-12-09 13:12:24,536::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3041::DEBUG::2012-12-09 13:12:24,536::task::978::TaskManager.Task::(_decref) Task=`b71e8dc3-8ab0-4940-89e0-850c06394f74`::ref 0 aborting False
>Thread-3042::DEBUG::2012-12-09 13:12:24,553::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3042::DEBUG::2012-12-09 13:12:24,554::task::588::TaskManager.Task::(_updateState) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::moving from state init -> state preparing
>Thread-3042::INFO::2012-12-09 13:12:24,554::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3042::DEBUG::2012-12-09 13:12:24,555::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a582361c-dbd8-4f62-a11f-06d6515bdef5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3042::DEBUG::2012-12-09 13:12:24,555::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3042::DEBUG::2012-12-09 13:12:24,555::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3042::DEBUG::2012-12-09 13:12:24,556::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a582361c-dbd8-4f62-a11f-06d6515bdef5`::Granted request
>Thread-3042::DEBUG::2012-12-09 13:12:24,556::task::817::TaskManager.Task::(resourceAcquired) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3042::DEBUG::2012-12-09 13:12:24,557::task::978::TaskManager.Task::(_decref) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::ref 1 aborting False
>Thread-3042::DEBUG::2012-12-09 13:12:24,557::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3042::DEBUG::2012-12-09 13:12:24,558::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3042::DEBUG::2012-12-09 13:12:24,657::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3042::WARNING::2012-12-09 13:12:24,658::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3042::DEBUG::2012-12-09 13:12:24,658::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3042::WARNING::2012-12-09 13:12:24,659::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3042::INFO::2012-12-09 13:12:24,659::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3042::DEBUG::2012-12-09 13:12:24,659::task::1172::TaskManager.Task::(prepare) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3042::DEBUG::2012-12-09 13:12:24,659::task::588::TaskManager.Task::(_updateState) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::moving from state preparing -> state finished
>Thread-3042::DEBUG::2012-12-09 13:12:24,660::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3042::DEBUG::2012-12-09 13:12:24,660::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3042::DEBUG::2012-12-09 13:12:24,660::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3042::DEBUG::2012-12-09 13:12:24,660::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3042::DEBUG::2012-12-09 13:12:24,660::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3042::DEBUG::2012-12-09 13:12:24,661::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3042::DEBUG::2012-12-09 13:12:24,661::task::978::TaskManager.Task::(_decref) Task=`0a2e677a-c893-4e74-9b42-7bb32c23e80f`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:12:24,936::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:24,937::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:25,046::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:25,048::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:25,048::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:25,049::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:12:25,825::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:25,867::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172838 s, 59.2 MB/s\n'; <rc> = 0
>Thread-3044::DEBUG::2012-12-09 13:12:27,585::task::588::TaskManager.Task::(_updateState) Task=`52831731-c294-4a34-acb3-859ba781635f`::moving from state init -> state preparing
>Thread-3044::INFO::2012-12-09 13:12:27,586::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3044::INFO::2012-12-09 13:12:27,586::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122890472412', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-3044::DEBUG::2012-12-09 13:12:27,586::task::1172::TaskManager.Task::(prepare) Task=`52831731-c294-4a34-acb3-859ba781635f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122890472412', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-3044::DEBUG::2012-12-09 13:12:27,586::task::588::TaskManager.Task::(_updateState) Task=`52831731-c294-4a34-acb3-859ba781635f`::moving from state preparing -> state finished
>Thread-3044::DEBUG::2012-12-09 13:12:27,587::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3044::DEBUG::2012-12-09 13:12:27,587::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3044::DEBUG::2012-12-09 13:12:27,587::task::978::TaskManager.Task::(_decref) Task=`52831731-c294-4a34-acb3-859ba781635f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:27,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:27,912::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159944 s, 64.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:29,918::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:29,957::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162753 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:31,964::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:32,004::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166651 s, 61.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:12:33,928::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:33,943::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00050568 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:34,011::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:34,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0290572 s, 35.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:12:35,050::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:35,051::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:35,153::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:35,154::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:35,154::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:35,155::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3049::DEBUG::2012-12-09 13:12:35,304::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3049::DEBUG::2012-12-09 13:12:35,304::task::588::TaskManager.Task::(_updateState) Task=`4e71711a-e63b-475f-9be1-10e90fcd59e8`::moving from state init -> state preparing
>Thread-3049::INFO::2012-12-09 13:12:35,304::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3049::INFO::2012-12-09 13:12:35,305::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3049::DEBUG::2012-12-09 13:12:35,305::task::1172::TaskManager.Task::(prepare) Task=`4e71711a-e63b-475f-9be1-10e90fcd59e8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3049::DEBUG::2012-12-09 13:12:35,305::task::588::TaskManager.Task::(_updateState) Task=`4e71711a-e63b-475f-9be1-10e90fcd59e8`::moving from state preparing -> state finished
>Thread-3049::DEBUG::2012-12-09 13:12:35,305::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3049::DEBUG::2012-12-09 13:12:35,305::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3049::DEBUG::2012-12-09 13:12:35,306::task::978::TaskManager.Task::(_decref) Task=`4e71711a-e63b-475f-9be1-10e90fcd59e8`::ref 0 aborting False
>Thread-3050::DEBUG::2012-12-09 13:12:35,317::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3050::DEBUG::2012-12-09 13:12:35,318::task::588::TaskManager.Task::(_updateState) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::moving from state init -> state preparing
>Thread-3050::INFO::2012-12-09 13:12:35,318::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3050::DEBUG::2012-12-09 13:12:35,318::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fa6b45c4-161a-4b51-be2b-52ac48fbfc3f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3050::DEBUG::2012-12-09 13:12:35,318::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3050::DEBUG::2012-12-09 13:12:35,319::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3050::DEBUG::2012-12-09 13:12:35,319::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`fa6b45c4-161a-4b51-be2b-52ac48fbfc3f`::Granted request
>Thread-3050::DEBUG::2012-12-09 13:12:35,319::task::817::TaskManager.Task::(resourceAcquired) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3050::DEBUG::2012-12-09 13:12:35,319::task::978::TaskManager.Task::(_decref) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::ref 1 aborting False
>Thread-3050::DEBUG::2012-12-09 13:12:35,320::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3050::DEBUG::2012-12-09 13:12:35,320::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3050::DEBUG::2012-12-09 13:12:35,418::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3050::WARNING::2012-12-09 13:12:35,419::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3050::DEBUG::2012-12-09 13:12:35,420::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3050::WARNING::2012-12-09 13:12:35,420::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3050::INFO::2012-12-09 13:12:35,421::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3050::DEBUG::2012-12-09 13:12:35,421::task::1172::TaskManager.Task::(prepare) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3050::DEBUG::2012-12-09 13:12:35,421::task::588::TaskManager.Task::(_updateState) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::moving from state preparing -> state finished
>Thread-3050::DEBUG::2012-12-09 13:12:35,422::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3050::DEBUG::2012-12-09 13:12:35,422::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3050::DEBUG::2012-12-09 13:12:35,422::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3050::DEBUG::2012-12-09 13:12:35,423::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3050::DEBUG::2012-12-09 13:12:35,423::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3050::DEBUG::2012-12-09 13:12:35,423::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3050::DEBUG::2012-12-09 13:12:35,424::task::978::TaskManager.Task::(_decref) Task=`d4cb76b2-6c2d-464a-acc9-b07cd7e19bda`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:36,089::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:36,128::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168528 s, 60.8 MB/s\n'; <rc> = 0
>Thread-3052::DEBUG::2012-12-09 13:12:37,895::task::588::TaskManager.Task::(_updateState) Task=`1c39bbdc-5eb1-46e4-9154-560df978e4fd`::moving from state init -> state preparing
>Thread-3052::INFO::2012-12-09 13:12:37,895::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3052::INFO::2012-12-09 13:12:37,896::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160219669342', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-3052::DEBUG::2012-12-09 13:12:37,896::task::1172::TaskManager.Task::(prepare) Task=`1c39bbdc-5eb1-46e4-9154-560df978e4fd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160219669342', 'lastCheck': '3.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.7', 'code': 358, 'valid': False}}
>Thread-3052::DEBUG::2012-12-09 13:12:37,896::task::588::TaskManager.Task::(_updateState) Task=`1c39bbdc-5eb1-46e4-9154-560df978e4fd`::moving from state preparing -> state finished
>Thread-3052::DEBUG::2012-12-09 13:12:37,896::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3052::DEBUG::2012-12-09 13:12:37,896::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3052::DEBUG::2012-12-09 13:12:37,897::task::978::TaskManager.Task::(_decref) Task=`1c39bbdc-5eb1-46e4-9154-560df978e4fd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:38,135::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:38,175::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171944 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:40,182::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:40,222::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171677 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:42,229::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:42,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185816 s, 55.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:12:43,948::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:43,960::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000613517 s, 6.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:44,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:44,328::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213548 s, 48.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:12:45,156::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:45,156::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:45,255::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:45,256::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:45,256::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:45,257::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3058::DEBUG::2012-12-09 13:12:46,225::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3058::DEBUG::2012-12-09 13:12:46,226::task::588::TaskManager.Task::(_updateState) Task=`05ae04a6-e4a6-4623-aad7-8d500ad67214`::moving from state init -> state preparing
>Thread-3058::INFO::2012-12-09 13:12:46,226::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3058::INFO::2012-12-09 13:12:46,227::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3058::DEBUG::2012-12-09 13:12:46,227::task::1172::TaskManager.Task::(prepare) Task=`05ae04a6-e4a6-4623-aad7-8d500ad67214`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3058::DEBUG::2012-12-09 13:12:46,228::task::588::TaskManager.Task::(_updateState) Task=`05ae04a6-e4a6-4623-aad7-8d500ad67214`::moving from state preparing -> state finished
>Thread-3058::DEBUG::2012-12-09 13:12:46,228::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3058::DEBUG::2012-12-09 13:12:46,228::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3058::DEBUG::2012-12-09 13:12:46,229::task::978::TaskManager.Task::(_decref) Task=`05ae04a6-e4a6-4623-aad7-8d500ad67214`::ref 0 aborting False
>Thread-3059::DEBUG::2012-12-09 13:12:46,246::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3059::DEBUG::2012-12-09 13:12:46,247::task::588::TaskManager.Task::(_updateState) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::moving from state init -> state preparing
>Thread-3059::INFO::2012-12-09 13:12:46,247::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3059::DEBUG::2012-12-09 13:12:46,248::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76c14c6c-e2ba-49b4-a8e6-40fddb46732c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3059::DEBUG::2012-12-09 13:12:46,248::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3059::DEBUG::2012-12-09 13:12:46,248::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3059::DEBUG::2012-12-09 13:12:46,248::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`76c14c6c-e2ba-49b4-a8e6-40fddb46732c`::Granted request
>Thread-3059::DEBUG::2012-12-09 13:12:46,248::task::817::TaskManager.Task::(resourceAcquired) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3059::DEBUG::2012-12-09 13:12:46,249::task::978::TaskManager.Task::(_decref) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::ref 1 aborting False
>Thread-3059::DEBUG::2012-12-09 13:12:46,249::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3059::DEBUG::2012-12-09 13:12:46,250::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:46,335::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3059::DEBUG::2012-12-09 13:12:46,349::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3059::WARNING::2012-12-09 13:12:46,352::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3059::DEBUG::2012-12-09 13:12:46,352::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3059::WARNING::2012-12-09 13:12:46,354::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3059::INFO::2012-12-09 13:12:46,354::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3059::DEBUG::2012-12-09 13:12:46,355::task::1172::TaskManager.Task::(prepare) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3059::DEBUG::2012-12-09 13:12:46,355::task::588::TaskManager.Task::(_updateState) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::moving from state preparing -> state finished
>Thread-3059::DEBUG::2012-12-09 13:12:46,356::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3059::DEBUG::2012-12-09 13:12:46,356::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3059::DEBUG::2012-12-09 13:12:46,357::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3059::DEBUG::2012-12-09 13:12:46,357::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3059::DEBUG::2012-12-09 13:12:46,358::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3059::DEBUG::2012-12-09 13:12:46,358::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3059::DEBUG::2012-12-09 13:12:46,358::task::978::TaskManager.Task::(_decref) Task=`e608f7bd-85f9-4a0f-87e3-aafecf7b3d4e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:46,392::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282203 s, 36.3 MB/s\n'; <rc> = 0
>Thread-3060::DEBUG::2012-12-09 13:12:48,167::task::588::TaskManager.Task::(_updateState) Task=`3d4ade1c-c107-4194-ae5f-10ed793da14f`::moving from state init -> state preparing
>Thread-3060::INFO::2012-12-09 13:12:48,168::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3060::INFO::2012-12-09 13:12:48,168::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117809772491', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-3060::DEBUG::2012-12-09 13:12:48,168::task::1172::TaskManager.Task::(prepare) Task=`3d4ade1c-c107-4194-ae5f-10ed793da14f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117809772491', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-3060::DEBUG::2012-12-09 13:12:48,169::task::588::TaskManager.Task::(_updateState) Task=`3d4ade1c-c107-4194-ae5f-10ed793da14f`::moving from state preparing -> state finished
>Thread-3060::DEBUG::2012-12-09 13:12:48,169::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3060::DEBUG::2012-12-09 13:12:48,169::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3060::DEBUG::2012-12-09 13:12:48,170::task::978::TaskManager.Task::(_decref) Task=`3d4ade1c-c107-4194-ae5f-10ed793da14f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:48,398::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:48,463::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259218 s, 39.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:50,472::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:50,514::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183127 s, 55.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:52,521::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:52,562::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165447 s, 61.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:12:53,963::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:12:53,971::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00059742 s, 6.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:12:54,569::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:54,608::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161087 s, 63.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:12:55,258::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:12:55,259::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:12:55,364::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:12:55,365::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:12:55,365::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:12:55,365::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:12:56,614::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:56,654::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158596 s, 64.6 MB/s\n'; <rc> = 0
>Thread-3066::DEBUG::2012-12-09 13:12:57,346::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3066::DEBUG::2012-12-09 13:12:57,347::task::588::TaskManager.Task::(_updateState) Task=`f2b8cc21-c300-492e-98c5-2ed933449058`::moving from state init -> state preparing
>Thread-3066::INFO::2012-12-09 13:12:57,347::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3066::INFO::2012-12-09 13:12:57,347::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3066::DEBUG::2012-12-09 13:12:57,348::task::1172::TaskManager.Task::(prepare) Task=`f2b8cc21-c300-492e-98c5-2ed933449058`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3066::DEBUG::2012-12-09 13:12:57,348::task::588::TaskManager.Task::(_updateState) Task=`f2b8cc21-c300-492e-98c5-2ed933449058`::moving from state preparing -> state finished
>Thread-3066::DEBUG::2012-12-09 13:12:57,348::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3066::DEBUG::2012-12-09 13:12:57,348::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3066::DEBUG::2012-12-09 13:12:57,348::task::978::TaskManager.Task::(_decref) Task=`f2b8cc21-c300-492e-98c5-2ed933449058`::ref 0 aborting False
>Thread-3067::DEBUG::2012-12-09 13:12:57,360::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3067::DEBUG::2012-12-09 13:12:57,360::task::588::TaskManager.Task::(_updateState) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::moving from state init -> state preparing
>Thread-3067::INFO::2012-12-09 13:12:57,361::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3067::DEBUG::2012-12-09 13:12:57,361::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5a8d8c55-d8e3-4c1e-9e73-b2dada4bf027`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3067::DEBUG::2012-12-09 13:12:57,361::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3067::DEBUG::2012-12-09 13:12:57,361::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3067::DEBUG::2012-12-09 13:12:57,362::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5a8d8c55-d8e3-4c1e-9e73-b2dada4bf027`::Granted request
>Thread-3067::DEBUG::2012-12-09 13:12:57,362::task::817::TaskManager.Task::(resourceAcquired) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3067::DEBUG::2012-12-09 13:12:57,362::task::978::TaskManager.Task::(_decref) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::ref 1 aborting False
>Thread-3067::DEBUG::2012-12-09 13:12:57,363::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3067::DEBUG::2012-12-09 13:12:57,363::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3067::DEBUG::2012-12-09 13:12:57,468::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3067::WARNING::2012-12-09 13:12:57,469::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3067::DEBUG::2012-12-09 13:12:57,469::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3067::WARNING::2012-12-09 13:12:57,470::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3067::INFO::2012-12-09 13:12:57,470::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3067::DEBUG::2012-12-09 13:12:57,470::task::1172::TaskManager.Task::(prepare) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3067::DEBUG::2012-12-09 13:12:57,470::task::588::TaskManager.Task::(_updateState) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::moving from state preparing -> state finished
>Thread-3067::DEBUG::2012-12-09 13:12:57,471::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3067::DEBUG::2012-12-09 13:12:57,471::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3067::DEBUG::2012-12-09 13:12:57,471::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3067::DEBUG::2012-12-09 13:12:57,471::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3067::DEBUG::2012-12-09 13:12:57,471::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3067::DEBUG::2012-12-09 13:12:57,472::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3067::DEBUG::2012-12-09 13:12:57,472::task::978::TaskManager.Task::(_decref) Task=`a5834207-65c6-4508-9438-50623e9ab22a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:12:58,661::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:12:58,701::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170862 s, 59.9 MB/s\n'; <rc> = 0
>Thread-3068::DEBUG::2012-12-09 13:12:58,724::task::588::TaskManager.Task::(_updateState) Task=`6beebacc-c624-4bc8-9d3c-098d7994747c`::moving from state init -> state preparing
>Thread-3068::INFO::2012-12-09 13:12:58,724::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3068::INFO::2012-12-09 13:12:58,725::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00911092758179', 'lastCheck': '4.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-3068::DEBUG::2012-12-09 13:12:58,725::task::1172::TaskManager.Task::(prepare) Task=`6beebacc-c624-4bc8-9d3c-098d7994747c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00911092758179', 'lastCheck': '4.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.4', 'code': 358, 'valid': False}}
>Thread-3068::DEBUG::2012-12-09 13:12:58,725::task::588::TaskManager.Task::(_updateState) Task=`6beebacc-c624-4bc8-9d3c-098d7994747c`::moving from state preparing -> state finished
>Thread-3068::DEBUG::2012-12-09 13:12:58,725::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3068::DEBUG::2012-12-09 13:12:58,726::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3068::DEBUG::2012-12-09 13:12:58,726::task::978::TaskManager.Task::(_decref) Task=`6beebacc-c624-4bc8-9d3c-098d7994747c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:13:00,708::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:00,751::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167656 s, 61.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:02,758::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:02,820::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222936 s, 45.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:13:03,975::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:03,987::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00036012 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:04,826::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:04,870::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192459 s, 53.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:13:05,366::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:05,367::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:05,473::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:05,475::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:05,476::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:05,476::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:06,877::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:06,932::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259501 s, 39.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:08,939::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:08,979::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167389 s, 61.2 MB/s\n'; <rc> = 0
>Thread-3074::DEBUG::2012-12-09 13:13:10,401::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3074::DEBUG::2012-12-09 13:13:10,402::task::588::TaskManager.Task::(_updateState) Task=`6a4ba5c5-fab7-4aeb-a340-3626fd28f185`::moving from state init -> state preparing
>Thread-3074::INFO::2012-12-09 13:13:10,402::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3074::INFO::2012-12-09 13:13:10,403::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3074::DEBUG::2012-12-09 13:13:10,403::task::1172::TaskManager.Task::(prepare) Task=`6a4ba5c5-fab7-4aeb-a340-3626fd28f185`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3074::DEBUG::2012-12-09 13:13:10,403::task::588::TaskManager.Task::(_updateState) Task=`6a4ba5c5-fab7-4aeb-a340-3626fd28f185`::moving from state preparing -> state finished
>Thread-3074::DEBUG::2012-12-09 13:13:10,404::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3074::DEBUG::2012-12-09 13:13:10,404::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3074::DEBUG::2012-12-09 13:13:10,404::task::978::TaskManager.Task::(_decref) Task=`6a4ba5c5-fab7-4aeb-a340-3626fd28f185`::ref 0 aborting False
>Thread-3075::DEBUG::2012-12-09 13:13:10,423::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3075::DEBUG::2012-12-09 13:13:10,423::task::588::TaskManager.Task::(_updateState) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::moving from state init -> state preparing
>Thread-3075::INFO::2012-12-09 13:13:10,423::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3075::DEBUG::2012-12-09 13:13:10,424::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`03bbec30-fcbb-4ac9-9429-faf3bb8be361`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3075::DEBUG::2012-12-09 13:13:10,424::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3075::DEBUG::2012-12-09 13:13:10,424::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3075::DEBUG::2012-12-09 13:13:10,424::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`03bbec30-fcbb-4ac9-9429-faf3bb8be361`::Granted request
>Thread-3075::DEBUG::2012-12-09 13:13:10,425::task::817::TaskManager.Task::(resourceAcquired) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3075::DEBUG::2012-12-09 13:13:10,425::task::978::TaskManager.Task::(_decref) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::ref 1 aborting False
>Thread-3075::DEBUG::2012-12-09 13:13:10,426::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3075::DEBUG::2012-12-09 13:13:10,426::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3075::DEBUG::2012-12-09 13:13:10,560::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3075::WARNING::2012-12-09 13:13:10,562::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3075::DEBUG::2012-12-09 13:13:10,562::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3075::WARNING::2012-12-09 13:13:10,563::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3075::INFO::2012-12-09 13:13:10,564::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3075::DEBUG::2012-12-09 13:13:10,564::task::1172::TaskManager.Task::(prepare) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3075::DEBUG::2012-12-09 13:13:10,564::task::588::TaskManager.Task::(_updateState) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::moving from state preparing -> state finished
>Thread-3075::DEBUG::2012-12-09 13:13:10,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3075::DEBUG::2012-12-09 13:13:10,565::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3075::DEBUG::2012-12-09 13:13:10,565::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3075::DEBUG::2012-12-09 13:13:10,566::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3075::DEBUG::2012-12-09 13:13:10,566::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3075::DEBUG::2012-12-09 13:13:10,566::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3075::DEBUG::2012-12-09 13:13:10,567::task::978::TaskManager.Task::(_decref) Task=`369e34ff-c6a4-4c5f-8389-0e0637f3823f`::ref 0 aborting False
>Thread-3076::DEBUG::2012-12-09 13:13:10,766::task::588::TaskManager.Task::(_updateState) Task=`5b73264b-45e7-4b88-b022-db7953ed08e2`::moving from state init -> state preparing
>Thread-3076::INFO::2012-12-09 13:13:10,766::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3076::INFO::2012-12-09 13:13:10,767::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127811431885', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-3076::DEBUG::2012-12-09 13:13:10,767::task::1172::TaskManager.Task::(prepare) Task=`5b73264b-45e7-4b88-b022-db7953ed08e2`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127811431885', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-3076::DEBUG::2012-12-09 13:13:10,767::task::588::TaskManager.Task::(_updateState) Task=`5b73264b-45e7-4b88-b022-db7953ed08e2`::moving from state preparing -> state finished
>Thread-3076::DEBUG::2012-12-09 13:13:10,767::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3076::DEBUG::2012-12-09 13:13:10,768::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3076::DEBUG::2012-12-09 13:13:10,768::task::978::TaskManager.Task::(_decref) Task=`5b73264b-45e7-4b88-b022-db7953ed08e2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:13:10,986::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:11,031::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195222 s, 52.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:13,038::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:13,109::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310828 s, 32.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:13:13,990::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:14,001::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000503077 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:15,119::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:15,165::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215661 s, 47.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:13:15,477::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:15,478::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:15,589::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:15,591::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:15,592::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:15,593::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:17,172::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:17,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179687 s, 57.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:19,223::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:19,264::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167764 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:21,271::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:21,315::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186294 s, 55.0 MB/s\n'; <rc> = 0
>Thread-3082::DEBUG::2012-12-09 13:13:21,957::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3082::DEBUG::2012-12-09 13:13:21,958::task::588::TaskManager.Task::(_updateState) Task=`fed42acd-9afe-421a-9ab4-166806590d92`::moving from state init -> state preparing
>Thread-3082::INFO::2012-12-09 13:13:21,958::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3082::INFO::2012-12-09 13:13:21,958::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3082::DEBUG::2012-12-09 13:13:21,958::task::1172::TaskManager.Task::(prepare) Task=`fed42acd-9afe-421a-9ab4-166806590d92`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3082::DEBUG::2012-12-09 13:13:21,959::task::588::TaskManager.Task::(_updateState) Task=`fed42acd-9afe-421a-9ab4-166806590d92`::moving from state preparing -> state finished
>Thread-3082::DEBUG::2012-12-09 13:13:21,959::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3082::DEBUG::2012-12-09 13:13:21,959::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3082::DEBUG::2012-12-09 13:13:21,959::task::978::TaskManager.Task::(_decref) Task=`fed42acd-9afe-421a-9ab4-166806590d92`::ref 0 aborting False
>Thread-3083::DEBUG::2012-12-09 13:13:21,975::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3083::DEBUG::2012-12-09 13:13:21,975::task::588::TaskManager.Task::(_updateState) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::moving from state init -> state preparing
>Thread-3083::INFO::2012-12-09 13:13:21,976::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3083::DEBUG::2012-12-09 13:13:21,976::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6d1cbe72-8446-448b-a1dd-78698c761da6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3083::DEBUG::2012-12-09 13:13:21,976::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3083::DEBUG::2012-12-09 13:13:21,976::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3083::DEBUG::2012-12-09 13:13:21,977::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6d1cbe72-8446-448b-a1dd-78698c761da6`::Granted request
>Thread-3083::DEBUG::2012-12-09 13:13:21,977::task::817::TaskManager.Task::(resourceAcquired) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3083::DEBUG::2012-12-09 13:13:21,977::task::978::TaskManager.Task::(_decref) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::ref 1 aborting False
>Thread-3083::DEBUG::2012-12-09 13:13:21,979::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3083::DEBUG::2012-12-09 13:13:21,979::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3084::DEBUG::2012-12-09 13:13:22,048::task::588::TaskManager.Task::(_updateState) Task=`5d0e4998-fcf0-4a5f-be9d-f3a4cb96bb69`::moving from state init -> state preparing
>Thread-3084::INFO::2012-12-09 13:13:22,049::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3084::INFO::2012-12-09 13:13:22,049::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.01189494133', 'lastCheck': '8.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3084::DEBUG::2012-12-09 13:13:22,049::task::1172::TaskManager.Task::(prepare) Task=`5d0e4998-fcf0-4a5f-be9d-f3a4cb96bb69`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.01189494133', 'lastCheck': '8.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3084::DEBUG::2012-12-09 13:13:22,050::task::588::TaskManager.Task::(_updateState) Task=`5d0e4998-fcf0-4a5f-be9d-f3a4cb96bb69`::moving from state preparing -> state finished
>Thread-3084::DEBUG::2012-12-09 13:13:22,050::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3084::DEBUG::2012-12-09 13:13:22,050::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3084::DEBUG::2012-12-09 13:13:22,050::task::978::TaskManager.Task::(_decref) Task=`5d0e4998-fcf0-4a5f-be9d-f3a4cb96bb69`::ref 0 aborting False
>Thread-3083::DEBUG::2012-12-09 13:13:22,109::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3083::WARNING::2012-12-09 13:13:22,110::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3083::DEBUG::2012-12-09 13:13:22,111::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3083::WARNING::2012-12-09 13:13:22,111::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3083::INFO::2012-12-09 13:13:22,112::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3083::DEBUG::2012-12-09 13:13:22,112::task::1172::TaskManager.Task::(prepare) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3083::DEBUG::2012-12-09 13:13:22,113::task::588::TaskManager.Task::(_updateState) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::moving from state preparing -> state finished
>Thread-3083::DEBUG::2012-12-09 13:13:22,113::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3083::DEBUG::2012-12-09 13:13:22,113::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3083::DEBUG::2012-12-09 13:13:22,113::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3083::DEBUG::2012-12-09 13:13:22,114::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3083::DEBUG::2012-12-09 13:13:22,114::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3083::DEBUG::2012-12-09 13:13:22,114::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3083::DEBUG::2012-12-09 13:13:22,114::task::978::TaskManager.Task::(_decref) Task=`f51f0935-c879-4778-8d62-8128dd30f0aa`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:13:23,322::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:23,389::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028023 s, 36.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:13:24,005::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:24,018::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000425123 s, 9.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:25,399::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:25,443::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187996 s, 54.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:13:25,594::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:25,595::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:25,728::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:25,729::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:25,730::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:25,730::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:27,450::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:27,492::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167808 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:29,501::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:29,544::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181949 s, 56.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:31,550::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:31,616::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266012 s, 38.5 MB/s\n'; <rc> = 0
>Thread-3090::DEBUG::2012-12-09 13:13:32,365::task::588::TaskManager.Task::(_updateState) Task=`056db97c-6e4d-4b8e-87fe-6c28de032c0b`::moving from state init -> state preparing
>Thread-3090::INFO::2012-12-09 13:13:32,366::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3090::INFO::2012-12-09 13:13:32,366::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136711597443', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-3090::DEBUG::2012-12-09 13:13:32,366::task::1172::TaskManager.Task::(prepare) Task=`056db97c-6e4d-4b8e-87fe-6c28de032c0b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136711597443', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False}}
>Thread-3090::DEBUG::2012-12-09 13:13:32,366::task::588::TaskManager.Task::(_updateState) Task=`056db97c-6e4d-4b8e-87fe-6c28de032c0b`::moving from state preparing -> state finished
>Thread-3090::DEBUG::2012-12-09 13:13:32,366::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3090::DEBUG::2012-12-09 13:13:32,367::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3090::DEBUG::2012-12-09 13:13:32,367::task::978::TaskManager.Task::(_decref) Task=`056db97c-6e4d-4b8e-87fe-6c28de032c0b`::ref 0 aborting False
>Thread-3092::DEBUG::2012-12-09 13:13:32,458::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3092::DEBUG::2012-12-09 13:13:32,459::task::588::TaskManager.Task::(_updateState) Task=`323910c0-e2bb-4660-8be7-c5869bb0bece`::moving from state init -> state preparing
>Thread-3092::INFO::2012-12-09 13:13:32,459::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3092::INFO::2012-12-09 13:13:32,459::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3092::DEBUG::2012-12-09 13:13:32,460::task::1172::TaskManager.Task::(prepare) Task=`323910c0-e2bb-4660-8be7-c5869bb0bece`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3092::DEBUG::2012-12-09 13:13:32,460::task::588::TaskManager.Task::(_updateState) Task=`323910c0-e2bb-4660-8be7-c5869bb0bece`::moving from state preparing -> state finished
>Thread-3092::DEBUG::2012-12-09 13:13:32,460::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3092::DEBUG::2012-12-09 13:13:32,460::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3092::DEBUG::2012-12-09 13:13:32,460::task::978::TaskManager.Task::(_decref) Task=`323910c0-e2bb-4660-8be7-c5869bb0bece`::ref 0 aborting False
>Thread-3093::DEBUG::2012-12-09 13:13:32,473::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3093::DEBUG::2012-12-09 13:13:32,473::task::588::TaskManager.Task::(_updateState) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::moving from state init -> state preparing
>Thread-3093::INFO::2012-12-09 13:13:32,474::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3093::DEBUG::2012-12-09 13:13:32,474::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5df95a2d-41a1-4169-8ceb-53e96561f1fc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3093::DEBUG::2012-12-09 13:13:32,474::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3093::DEBUG::2012-12-09 13:13:32,474::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3093::DEBUG::2012-12-09 13:13:32,475::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5df95a2d-41a1-4169-8ceb-53e96561f1fc`::Granted request
>Thread-3093::DEBUG::2012-12-09 13:13:32,475::task::817::TaskManager.Task::(resourceAcquired) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3093::DEBUG::2012-12-09 13:13:32,475::task::978::TaskManager.Task::(_decref) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::ref 1 aborting False
>Thread-3093::DEBUG::2012-12-09 13:13:32,476::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3093::DEBUG::2012-12-09 13:13:32,476::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3093::DEBUG::2012-12-09 13:13:32,585::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3093::WARNING::2012-12-09 13:13:32,586::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3093::DEBUG::2012-12-09 13:13:32,587::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3093::WARNING::2012-12-09 13:13:32,587::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3093::INFO::2012-12-09 13:13:32,588::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3093::DEBUG::2012-12-09 13:13:32,588::task::1172::TaskManager.Task::(prepare) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3093::DEBUG::2012-12-09 13:13:32,588::task::588::TaskManager.Task::(_updateState) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::moving from state preparing -> state finished
>Thread-3093::DEBUG::2012-12-09 13:13:32,589::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3093::DEBUG::2012-12-09 13:13:32,589::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3093::DEBUG::2012-12-09 13:13:32,589::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3093::DEBUG::2012-12-09 13:13:32,590::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3093::DEBUG::2012-12-09 13:13:32,590::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3093::DEBUG::2012-12-09 13:13:32,590::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3093::DEBUG::2012-12-09 13:13:32,590::task::978::TaskManager.Task::(_decref) Task=`f8aa7e97-5477-4383-ab88-a17f1b462192`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:13:33,625::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:33,666::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170191 s, 60.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:13:34,021::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:34,032::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000504256 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:35,672::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:35,716::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194694 s, 52.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:13:35,731::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:35,732::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:35,840::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:35,841::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:35,841::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:35,841::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:37,724::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:37,773::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0213214 s, 48.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:39,781::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:39,825::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189868 s, 53.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:41,830::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:41,890::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269988 s, 37.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:43,905::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:43,947::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179684 s, 57.0 MB/s\n'; <rc> = 0
>Thread-3098::DEBUG::2012-12-09 13:13:44,015::task::588::TaskManager.Task::(_updateState) Task=`5500559d-96d5-4621-be0c-d5df054fe889`::moving from state init -> state preparing
>Thread-3098::INFO::2012-12-09 13:13:44,016::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3098::INFO::2012-12-09 13:13:44,016::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117218494415', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-3098::DEBUG::2012-12-09 13:13:44,016::task::1172::TaskManager.Task::(prepare) Task=`5500559d-96d5-4621-be0c-d5df054fe889`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117218494415', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.2', 'code': 358, 'valid': False}}
>Thread-3098::DEBUG::2012-12-09 13:13:44,016::task::588::TaskManager.Task::(_updateState) Task=`5500559d-96d5-4621-be0c-d5df054fe889`::moving from state preparing -> state finished
>Thread-3098::DEBUG::2012-12-09 13:13:44,017::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3098::DEBUG::2012-12-09 13:13:44,017::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3098::DEBUG::2012-12-09 13:13:44,017::task::978::TaskManager.Task::(_decref) Task=`5500559d-96d5-4621-be0c-d5df054fe889`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:13:44,037::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:44,045::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000357445 s, 11.5 MB/s\n'; <rc> = 0
>Thread-3100::DEBUG::2012-12-09 13:13:44,064::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3100::DEBUG::2012-12-09 13:13:44,065::task::588::TaskManager.Task::(_updateState) Task=`5859733c-d8cd-4a2a-87b4-64b12406f0d2`::moving from state init -> state preparing
>Thread-3100::INFO::2012-12-09 13:13:44,065::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3100::INFO::2012-12-09 13:13:44,066::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3100::DEBUG::2012-12-09 13:13:44,066::task::1172::TaskManager.Task::(prepare) Task=`5859733c-d8cd-4a2a-87b4-64b12406f0d2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3100::DEBUG::2012-12-09 13:13:44,066::task::588::TaskManager.Task::(_updateState) Task=`5859733c-d8cd-4a2a-87b4-64b12406f0d2`::moving from state preparing -> state finished
>Thread-3100::DEBUG::2012-12-09 13:13:44,066::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3100::DEBUG::2012-12-09 13:13:44,066::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3100::DEBUG::2012-12-09 13:13:44,067::task::978::TaskManager.Task::(_decref) Task=`5859733c-d8cd-4a2a-87b4-64b12406f0d2`::ref 0 aborting False
>Thread-3101::DEBUG::2012-12-09 13:13:44,079::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3101::DEBUG::2012-12-09 13:13:44,080::task::588::TaskManager.Task::(_updateState) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::moving from state init -> state preparing
>Thread-3101::INFO::2012-12-09 13:13:44,080::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3101::DEBUG::2012-12-09 13:13:44,080::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5c503f23-94fb-46ef-8c02-3fe7c98c5922`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3101::DEBUG::2012-12-09 13:13:44,081::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3101::DEBUG::2012-12-09 13:13:44,081::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3101::DEBUG::2012-12-09 13:13:44,081::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5c503f23-94fb-46ef-8c02-3fe7c98c5922`::Granted request
>Thread-3101::DEBUG::2012-12-09 13:13:44,081::task::817::TaskManager.Task::(resourceAcquired) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3101::DEBUG::2012-12-09 13:13:44,082::task::978::TaskManager.Task::(_decref) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::ref 1 aborting False
>Thread-3101::DEBUG::2012-12-09 13:13:44,082::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3101::DEBUG::2012-12-09 13:13:44,083::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3101::DEBUG::2012-12-09 13:13:44,180::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3101::WARNING::2012-12-09 13:13:44,181::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3101::DEBUG::2012-12-09 13:13:44,182::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3101::WARNING::2012-12-09 13:13:44,182::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3101::INFO::2012-12-09 13:13:44,183::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3101::DEBUG::2012-12-09 13:13:44,183::task::1172::TaskManager.Task::(prepare) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3101::DEBUG::2012-12-09 13:13:44,183::task::588::TaskManager.Task::(_updateState) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::moving from state preparing -> state finished
>Thread-3101::DEBUG::2012-12-09 13:13:44,183::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3101::DEBUG::2012-12-09 13:13:44,184::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3101::DEBUG::2012-12-09 13:13:44,184::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3101::DEBUG::2012-12-09 13:13:44,184::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3101::DEBUG::2012-12-09 13:13:44,184::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3101::DEBUG::2012-12-09 13:13:44,184::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3101::DEBUG::2012-12-09 13:13:44,185::task::978::TaskManager.Task::(_decref) Task=`f3256cbd-93e5-4215-8b1e-3fc99230d8d5`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:13:45,842::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:45,843::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:45,942::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:45,943::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:45,943::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:45,944::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:45,954::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:45,995::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172841 s, 59.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:48,002::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:48,042::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156847 s, 65.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:50,049::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:50,096::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171457 s, 59.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:52,106::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:52,148::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183511 s, 55.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:13:54,048::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:13:54,057::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00042191 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:54,155::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:54,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01809 s, 56.6 MB/s\n'; <rc> = 0
>Thread-3106::DEBUG::2012-12-09 13:13:54,992::task::588::TaskManager.Task::(_updateState) Task=`ecdd7d30-71a6-4d58-bf9d-7387d3975215`::moving from state init -> state preparing
>Thread-3106::INFO::2012-12-09 13:13:54,993::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3106::INFO::2012-12-09 13:13:54,994::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00939798355103', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.0', 'code': 358, 'valid': False}}
>Thread-3106::DEBUG::2012-12-09 13:13:54,994::task::1172::TaskManager.Task::(prepare) Task=`ecdd7d30-71a6-4d58-bf9d-7387d3975215`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00939798355103', 'lastCheck': '0.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.0', 'code': 358, 'valid': False}}
>Thread-3106::DEBUG::2012-12-09 13:13:54,995::task::588::TaskManager.Task::(_updateState) Task=`ecdd7d30-71a6-4d58-bf9d-7387d3975215`::moving from state preparing -> state finished
>Thread-3106::DEBUG::2012-12-09 13:13:54,995::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3106::DEBUG::2012-12-09 13:13:54,996::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3106::DEBUG::2012-12-09 13:13:54,996::task::978::TaskManager.Task::(_decref) Task=`ecdd7d30-71a6-4d58-bf9d-7387d3975215`::ref 0 aborting False
>Thread-3108::DEBUG::2012-12-09 13:13:55,278::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3108::DEBUG::2012-12-09 13:13:55,278::task::588::TaskManager.Task::(_updateState) Task=`e4c03ff8-b800-43f9-b89e-9fa34ac24fa8`::moving from state init -> state preparing
>Thread-3108::INFO::2012-12-09 13:13:55,279::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3108::INFO::2012-12-09 13:13:55,279::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3108::DEBUG::2012-12-09 13:13:55,279::task::1172::TaskManager.Task::(prepare) Task=`e4c03ff8-b800-43f9-b89e-9fa34ac24fa8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3108::DEBUG::2012-12-09 13:13:55,280::task::588::TaskManager.Task::(_updateState) Task=`e4c03ff8-b800-43f9-b89e-9fa34ac24fa8`::moving from state preparing -> state finished
>Thread-3108::DEBUG::2012-12-09 13:13:55,280::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3108::DEBUG::2012-12-09 13:13:55,280::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3108::DEBUG::2012-12-09 13:13:55,280::task::978::TaskManager.Task::(_decref) Task=`e4c03ff8-b800-43f9-b89e-9fa34ac24fa8`::ref 0 aborting False
>Thread-3109::DEBUG::2012-12-09 13:13:55,294::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3109::DEBUG::2012-12-09 13:13:55,294::task::588::TaskManager.Task::(_updateState) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::moving from state init -> state preparing
>Thread-3109::INFO::2012-12-09 13:13:55,294::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3109::DEBUG::2012-12-09 13:13:55,295::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e5492e8b-1cb8-43e7-8a49-8237cbd213ae`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3109::DEBUG::2012-12-09 13:13:55,295::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3109::DEBUG::2012-12-09 13:13:55,295::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3109::DEBUG::2012-12-09 13:13:55,296::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e5492e8b-1cb8-43e7-8a49-8237cbd213ae`::Granted request
>Thread-3109::DEBUG::2012-12-09 13:13:55,297::task::817::TaskManager.Task::(resourceAcquired) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3109::DEBUG::2012-12-09 13:13:55,297::task::978::TaskManager.Task::(_decref) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::ref 1 aborting False
>Thread-3109::DEBUG::2012-12-09 13:13:55,298::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3109::DEBUG::2012-12-09 13:13:55,298::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3109::DEBUG::2012-12-09 13:13:55,402::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3109::WARNING::2012-12-09 13:13:55,404::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3109::DEBUG::2012-12-09 13:13:55,404::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3109::WARNING::2012-12-09 13:13:55,404::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3109::INFO::2012-12-09 13:13:55,405::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3109::DEBUG::2012-12-09 13:13:55,405::task::1172::TaskManager.Task::(prepare) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3109::DEBUG::2012-12-09 13:13:55,405::task::588::TaskManager.Task::(_updateState) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::moving from state preparing -> state finished
>Thread-3109::DEBUG::2012-12-09 13:13:55,405::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3109::DEBUG::2012-12-09 13:13:55,406::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3109::DEBUG::2012-12-09 13:13:55,406::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3109::DEBUG::2012-12-09 13:13:55,406::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3109::DEBUG::2012-12-09 13:13:55,406::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3109::DEBUG::2012-12-09 13:13:55,406::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3109::DEBUG::2012-12-09 13:13:55,407::task::978::TaskManager.Task::(_decref) Task=`f12e4c45-3e7e-488e-b9ee-1897c0ea08f2`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:13:55,944::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:13:55,945::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:13:56,043::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:13:56,045::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:13:56,045::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:13:56,046::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:13:56,203::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:56,245::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168258 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:13:58,252::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:13:58,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163547 s, 62.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:00,299::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:00,340::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158958 s, 64.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:02,347::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:02,387::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171119 s, 59.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:04,060::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:04,068::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000480439 s, 8.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:04,395::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:04,437::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180882 s, 56.6 MB/s\n'; <rc> = 0
>Thread-3114::DEBUG::2012-12-09 13:14:05,844::task::588::TaskManager.Task::(_updateState) Task=`64085ed6-d2c9-48bd-839e-f76d6052abf9`::moving from state init -> state preparing
>Thread-3114::INFO::2012-12-09 13:14:05,844::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3114::INFO::2012-12-09 13:14:05,845::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839996337891', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-3114::DEBUG::2012-12-09 13:14:05,845::task::1172::TaskManager.Task::(prepare) Task=`64085ed6-d2c9-48bd-839e-f76d6052abf9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839996337891', 'lastCheck': '1.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.8', 'code': 358, 'valid': False}}
>Thread-3114::DEBUG::2012-12-09 13:14:05,845::task::588::TaskManager.Task::(_updateState) Task=`64085ed6-d2c9-48bd-839e-f76d6052abf9`::moving from state preparing -> state finished
>Thread-3114::DEBUG::2012-12-09 13:14:05,845::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3114::DEBUG::2012-12-09 13:14:05,846::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3114::DEBUG::2012-12-09 13:14:05,846::task::978::TaskManager.Task::(_decref) Task=`64085ed6-d2c9-48bd-839e-f76d6052abf9`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:14:06,046::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:06,047::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:06,146::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:06,147::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:06,148::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:06,148::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3116::DEBUG::2012-12-09 13:14:06,207::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3116::DEBUG::2012-12-09 13:14:06,208::task::588::TaskManager.Task::(_updateState) Task=`1f800c77-79bc-4cb1-86c1-401293110310`::moving from state init -> state preparing
>Thread-3116::INFO::2012-12-09 13:14:06,208::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3116::INFO::2012-12-09 13:14:06,208::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3116::DEBUG::2012-12-09 13:14:06,209::task::1172::TaskManager.Task::(prepare) Task=`1f800c77-79bc-4cb1-86c1-401293110310`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3116::DEBUG::2012-12-09 13:14:06,209::task::588::TaskManager.Task::(_updateState) Task=`1f800c77-79bc-4cb1-86c1-401293110310`::moving from state preparing -> state finished
>Thread-3116::DEBUG::2012-12-09 13:14:06,209::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3116::DEBUG::2012-12-09 13:14:06,209::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3116::DEBUG::2012-12-09 13:14:06,210::task::978::TaskManager.Task::(_decref) Task=`1f800c77-79bc-4cb1-86c1-401293110310`::ref 0 aborting False
>Thread-3117::DEBUG::2012-12-09 13:14:06,223::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3117::DEBUG::2012-12-09 13:14:06,224::task::588::TaskManager.Task::(_updateState) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::moving from state init -> state preparing
>Thread-3117::INFO::2012-12-09 13:14:06,224::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3117::DEBUG::2012-12-09 13:14:06,224::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d0669712-392b-4db7-ae34-c5b45be0245f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3117::DEBUG::2012-12-09 13:14:06,225::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3117::DEBUG::2012-12-09 13:14:06,225::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3117::DEBUG::2012-12-09 13:14:06,225::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d0669712-392b-4db7-ae34-c5b45be0245f`::Granted request
>Thread-3117::DEBUG::2012-12-09 13:14:06,225::task::817::TaskManager.Task::(resourceAcquired) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3117::DEBUG::2012-12-09 13:14:06,226::task::978::TaskManager.Task::(_decref) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::ref 1 aborting False
>Thread-3117::DEBUG::2012-12-09 13:14:06,226::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3117::DEBUG::2012-12-09 13:14:06,227::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3117::DEBUG::2012-12-09 13:14:06,332::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3117::WARNING::2012-12-09 13:14:06,333::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3117::DEBUG::2012-12-09 13:14:06,333::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3117::WARNING::2012-12-09 13:14:06,334::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3117::INFO::2012-12-09 13:14:06,334::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3117::DEBUG::2012-12-09 13:14:06,335::task::1172::TaskManager.Task::(prepare) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3117::DEBUG::2012-12-09 13:14:06,335::task::588::TaskManager.Task::(_updateState) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::moving from state preparing -> state finished
>Thread-3117::DEBUG::2012-12-09 13:14:06,335::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3117::DEBUG::2012-12-09 13:14:06,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3117::DEBUG::2012-12-09 13:14:06,336::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3117::DEBUG::2012-12-09 13:14:06,336::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3117::DEBUG::2012-12-09 13:14:06,336::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3117::DEBUG::2012-12-09 13:14:06,336::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3117::DEBUG::2012-12-09 13:14:06,336::task::978::TaskManager.Task::(_decref) Task=`ce9f0f7b-69af-4d37-84a8-aa6227668b15`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:06,443::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:06,514::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276451 s, 37.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:08,524::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:08,570::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187926 s, 54.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:10,578::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:10,623::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186987 s, 54.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:12,630::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:12,696::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268386 s, 38.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:14,070::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 13:14:14,071::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:14:14,071::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:14,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000423225 s, 4.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:14,081::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000423225 s, 4.8 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 13:14:14,081::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 13:14:14,081::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 13:14:14,082::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:14:14,082::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,082::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,083::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,083::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,083::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,083::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:14,185::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:14,186::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:14:14,186::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 13:14:14,187::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 13:14:14,187::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 13:14:14,187::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:14:14,188::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:14,288::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:14,289::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:14,301::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000495903 s, 8.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:14,706::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:14,748::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179359 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:14:16,149::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:16,150::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:16,250::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:16,252::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:16,252::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:16,253::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3122::DEBUG::2012-12-09 13:14:16,346::task::588::TaskManager.Task::(_updateState) Task=`9553ac9b-fdcc-4d4e-8e6b-e991de7d62a5`::moving from state init -> state preparing
>Thread-3122::INFO::2012-12-09 13:14:16,347::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3122::INFO::2012-12-09 13:14:16,347::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129141807556', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-3122::DEBUG::2012-12-09 13:14:16,347::task::1172::TaskManager.Task::(prepare) Task=`9553ac9b-fdcc-4d4e-8e6b-e991de7d62a5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0129141807556', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-3122::DEBUG::2012-12-09 13:14:16,347::task::588::TaskManager.Task::(_updateState) Task=`9553ac9b-fdcc-4d4e-8e6b-e991de7d62a5`::moving from state preparing -> state finished
>Thread-3122::DEBUG::2012-12-09 13:14:16,348::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3122::DEBUG::2012-12-09 13:14:16,348::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3122::DEBUG::2012-12-09 13:14:16,348::task::978::TaskManager.Task::(_decref) Task=`9553ac9b-fdcc-4d4e-8e6b-e991de7d62a5`::ref 0 aborting False
>Thread-3124::DEBUG::2012-12-09 13:14:16,689::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3124::DEBUG::2012-12-09 13:14:16,690::task::588::TaskManager.Task::(_updateState) Task=`7cf2b8c7-e81e-4625-a356-85df2066c1d4`::moving from state init -> state preparing
>Thread-3124::INFO::2012-12-09 13:14:16,690::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3124::INFO::2012-12-09 13:14:16,690::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3124::DEBUG::2012-12-09 13:14:16,691::task::1172::TaskManager.Task::(prepare) Task=`7cf2b8c7-e81e-4625-a356-85df2066c1d4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3124::DEBUG::2012-12-09 13:14:16,691::task::588::TaskManager.Task::(_updateState) Task=`7cf2b8c7-e81e-4625-a356-85df2066c1d4`::moving from state preparing -> state finished
>Thread-3124::DEBUG::2012-12-09 13:14:16,691::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3124::DEBUG::2012-12-09 13:14:16,691::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3124::DEBUG::2012-12-09 13:14:16,692::task::978::TaskManager.Task::(_decref) Task=`7cf2b8c7-e81e-4625-a356-85df2066c1d4`::ref 0 aborting False
>Thread-3125::DEBUG::2012-12-09 13:14:16,707::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3125::DEBUG::2012-12-09 13:14:16,708::task::588::TaskManager.Task::(_updateState) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::moving from state init -> state preparing
>Thread-3125::INFO::2012-12-09 13:14:16,708::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3125::DEBUG::2012-12-09 13:14:16,709::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b235179d-dc51-4522-9806-e723e558ba12`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3125::DEBUG::2012-12-09 13:14:16,709::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3125::DEBUG::2012-12-09 13:14:16,710::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3125::DEBUG::2012-12-09 13:14:16,710::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b235179d-dc51-4522-9806-e723e558ba12`::Granted request
>Thread-3125::DEBUG::2012-12-09 13:14:16,710::task::817::TaskManager.Task::(resourceAcquired) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3125::DEBUG::2012-12-09 13:14:16,711::task::978::TaskManager.Task::(_decref) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::ref 1 aborting False
>Thread-3125::DEBUG::2012-12-09 13:14:16,712::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3125::DEBUG::2012-12-09 13:14:16,712::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:16,755::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:16,800::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184613 s, 55.5 MB/s\n'; <rc> = 0
>Thread-3125::DEBUG::2012-12-09 13:14:16,864::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3125::WARNING::2012-12-09 13:14:16,865::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3125::DEBUG::2012-12-09 13:14:16,866::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3125::WARNING::2012-12-09 13:14:16,867::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3125::INFO::2012-12-09 13:14:16,867::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3125::DEBUG::2012-12-09 13:14:16,868::task::1172::TaskManager.Task::(prepare) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3125::DEBUG::2012-12-09 13:14:16,868::task::588::TaskManager.Task::(_updateState) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::moving from state preparing -> state finished
>Thread-3125::DEBUG::2012-12-09 13:14:16,868::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3125::DEBUG::2012-12-09 13:14:16,869::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3125::DEBUG::2012-12-09 13:14:16,869::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3125::DEBUG::2012-12-09 13:14:16,870::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3125::DEBUG::2012-12-09 13:14:16,870::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3125::DEBUG::2012-12-09 13:14:16,870::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3125::DEBUG::2012-12-09 13:14:16,870::task::978::TaskManager.Task::(_decref) Task=`611d8a07-1eab-4e94-8bf1-a70e5c4c5d77`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:18,807::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:18,852::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182781 s, 56.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:20,859::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:20,900::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174241 s, 58.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:22,907::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:22,948::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166226 s, 61.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:24,305::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:24,317::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000451972 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:24,955::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:24,995::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162953 s, 62.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:14:26,253::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:26,254::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:26,369::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:26,370::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:26,371::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:26,371::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:14:27,001::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:27,042::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155843 s, 65.7 MB/s\n'; <rc> = 0
>Thread-3130::DEBUG::2012-12-09 13:14:27,726::task::588::TaskManager.Task::(_updateState) Task=`911b56e7-b125-4406-ae14-fc087209cbee`::moving from state init -> state preparing
>Thread-3130::INFO::2012-12-09 13:14:27,727::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3130::INFO::2012-12-09 13:14:27,727::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119738578796', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-3130::DEBUG::2012-12-09 13:14:27,727::task::1172::TaskManager.Task::(prepare) Task=`911b56e7-b125-4406-ae14-fc087209cbee`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119738578796', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.4', 'code': 358, 'valid': False}}
>Thread-3130::DEBUG::2012-12-09 13:14:27,727::task::588::TaskManager.Task::(_updateState) Task=`911b56e7-b125-4406-ae14-fc087209cbee`::moving from state preparing -> state finished
>Thread-3130::DEBUG::2012-12-09 13:14:27,727::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3130::DEBUG::2012-12-09 13:14:27,728::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3130::DEBUG::2012-12-09 13:14:27,728::task::978::TaskManager.Task::(_decref) Task=`911b56e7-b125-4406-ae14-fc087209cbee`::ref 0 aborting False
>Thread-3132::DEBUG::2012-12-09 13:14:28,066::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3132::DEBUG::2012-12-09 13:14:28,067::task::588::TaskManager.Task::(_updateState) Task=`bf73f097-efe0-4b5c-800f-8a51057edae2`::moving from state init -> state preparing
>Thread-3132::INFO::2012-12-09 13:14:28,067::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3132::INFO::2012-12-09 13:14:28,068::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3132::DEBUG::2012-12-09 13:14:28,068::task::1172::TaskManager.Task::(prepare) Task=`bf73f097-efe0-4b5c-800f-8a51057edae2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3132::DEBUG::2012-12-09 13:14:28,068::task::588::TaskManager.Task::(_updateState) Task=`bf73f097-efe0-4b5c-800f-8a51057edae2`::moving from state preparing -> state finished
>Thread-3132::DEBUG::2012-12-09 13:14:28,068::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3132::DEBUG::2012-12-09 13:14:28,068::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3132::DEBUG::2012-12-09 13:14:28,069::task::978::TaskManager.Task::(_decref) Task=`bf73f097-efe0-4b5c-800f-8a51057edae2`::ref 0 aborting False
>Thread-3133::DEBUG::2012-12-09 13:14:28,081::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3133::DEBUG::2012-12-09 13:14:28,082::task::588::TaskManager.Task::(_updateState) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::moving from state init -> state preparing
>Thread-3133::INFO::2012-12-09 13:14:28,082::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3133::DEBUG::2012-12-09 13:14:28,082::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0737d716-75bd-4df7-a27e-23d043b0e2dd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3133::DEBUG::2012-12-09 13:14:28,082::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3133::DEBUG::2012-12-09 13:14:28,083::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3133::DEBUG::2012-12-09 13:14:28,083::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0737d716-75bd-4df7-a27e-23d043b0e2dd`::Granted request
>Thread-3133::DEBUG::2012-12-09 13:14:28,083::task::817::TaskManager.Task::(resourceAcquired) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3133::DEBUG::2012-12-09 13:14:28,083::task::978::TaskManager.Task::(_decref) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::ref 1 aborting False
>Thread-3133::DEBUG::2012-12-09 13:14:28,084::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3133::DEBUG::2012-12-09 13:14:28,085::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3133::DEBUG::2012-12-09 13:14:28,184::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3133::WARNING::2012-12-09 13:14:28,185::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3133::DEBUG::2012-12-09 13:14:28,185::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3133::WARNING::2012-12-09 13:14:28,186::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3133::INFO::2012-12-09 13:14:28,186::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3133::DEBUG::2012-12-09 13:14:28,186::task::1172::TaskManager.Task::(prepare) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3133::DEBUG::2012-12-09 13:14:28,187::task::588::TaskManager.Task::(_updateState) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::moving from state preparing -> state finished
>Thread-3133::DEBUG::2012-12-09 13:14:28,187::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3133::DEBUG::2012-12-09 13:14:28,187::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3133::DEBUG::2012-12-09 13:14:28,187::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3133::DEBUG::2012-12-09 13:14:28,188::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3133::DEBUG::2012-12-09 13:14:28,188::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3133::DEBUG::2012-12-09 13:14:28,188::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3133::DEBUG::2012-12-09 13:14:28,188::task::978::TaskManager.Task::(_decref) Task=`71051fe2-b5d0-407a-a1aa-3607e6699e7c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:29,049::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:29,090::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176654 s, 58.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:31,097::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:31,140::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175675 s, 58.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:33,148::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:33,189::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171128 s, 59.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:34,320::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:34,331::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000497611 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:35,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:35,236::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167899 s, 61.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:14:36,372::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:36,373::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:36,474::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:36,475::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:36,475::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:36,476::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:14:37,243::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:37,288::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180073 s, 56.9 MB/s\n'; <rc> = 0
>Thread-3138::DEBUG::2012-12-09 13:14:38,021::task::588::TaskManager.Task::(_updateState) Task=`27aaf78c-d7d6-4c47-95b7-6c52e4a15eac`::moving from state init -> state preparing
>Thread-3138::INFO::2012-12-09 13:14:38,021::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3138::INFO::2012-12-09 13:14:38,022::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117840766907', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-3138::DEBUG::2012-12-09 13:14:38,022::task::1172::TaskManager.Task::(prepare) Task=`27aaf78c-d7d6-4c47-95b7-6c52e4a15eac`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0117840766907', 'lastCheck': '3.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-3138::DEBUG::2012-12-09 13:14:38,022::task::588::TaskManager.Task::(_updateState) Task=`27aaf78c-d7d6-4c47-95b7-6c52e4a15eac`::moving from state preparing -> state finished
>Thread-3138::DEBUG::2012-12-09 13:14:38,022::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3138::DEBUG::2012-12-09 13:14:38,022::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3138::DEBUG::2012-12-09 13:14:38,023::task::978::TaskManager.Task::(_decref) Task=`27aaf78c-d7d6-4c47-95b7-6c52e4a15eac`::ref 0 aborting False
>Thread-3140::DEBUG::2012-12-09 13:14:38,837::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3140::DEBUG::2012-12-09 13:14:38,838::task::588::TaskManager.Task::(_updateState) Task=`bda87151-570b-4524-8f33-9a072eaf0779`::moving from state init -> state preparing
>Thread-3140::INFO::2012-12-09 13:14:38,838::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3140::INFO::2012-12-09 13:14:38,839::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3140::DEBUG::2012-12-09 13:14:38,839::task::1172::TaskManager.Task::(prepare) Task=`bda87151-570b-4524-8f33-9a072eaf0779`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3140::DEBUG::2012-12-09 13:14:38,839::task::588::TaskManager.Task::(_updateState) Task=`bda87151-570b-4524-8f33-9a072eaf0779`::moving from state preparing -> state finished
>Thread-3140::DEBUG::2012-12-09 13:14:38,839::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3140::DEBUG::2012-12-09 13:14:38,839::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3140::DEBUG::2012-12-09 13:14:38,840::task::978::TaskManager.Task::(_decref) Task=`bda87151-570b-4524-8f33-9a072eaf0779`::ref 0 aborting False
>Thread-3141::DEBUG::2012-12-09 13:14:38,852::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3141::DEBUG::2012-12-09 13:14:38,853::task::588::TaskManager.Task::(_updateState) Task=`1c860795-9ef2-472f-928b-054736870ccc`::moving from state init -> state preparing
>Thread-3141::INFO::2012-12-09 13:14:38,853::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3141::DEBUG::2012-12-09 13:14:38,853::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df445c6b-d0ea-4a4d-bb98-992b705f1077`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3141::DEBUG::2012-12-09 13:14:38,854::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3141::DEBUG::2012-12-09 13:14:38,854::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3141::DEBUG::2012-12-09 13:14:38,854::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`df445c6b-d0ea-4a4d-bb98-992b705f1077`::Granted request
>Thread-3141::DEBUG::2012-12-09 13:14:38,854::task::817::TaskManager.Task::(resourceAcquired) Task=`1c860795-9ef2-472f-928b-054736870ccc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3141::DEBUG::2012-12-09 13:14:38,855::task::978::TaskManager.Task::(_decref) Task=`1c860795-9ef2-472f-928b-054736870ccc`::ref 1 aborting False
>Thread-3141::DEBUG::2012-12-09 13:14:38,855::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3141::DEBUG::2012-12-09 13:14:38,856::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3141::DEBUG::2012-12-09 13:14:38,958::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3141::WARNING::2012-12-09 13:14:38,959::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3141::DEBUG::2012-12-09 13:14:38,959::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3141::WARNING::2012-12-09 13:14:38,960::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3141::INFO::2012-12-09 13:14:38,960::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3141::DEBUG::2012-12-09 13:14:38,960::task::1172::TaskManager.Task::(prepare) Task=`1c860795-9ef2-472f-928b-054736870ccc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3141::DEBUG::2012-12-09 13:14:38,961::task::588::TaskManager.Task::(_updateState) Task=`1c860795-9ef2-472f-928b-054736870ccc`::moving from state preparing -> state finished
>Thread-3141::DEBUG::2012-12-09 13:14:38,961::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3141::DEBUG::2012-12-09 13:14:38,961::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3141::DEBUG::2012-12-09 13:14:38,961::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3141::DEBUG::2012-12-09 13:14:38,962::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3141::DEBUG::2012-12-09 13:14:38,962::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3141::DEBUG::2012-12-09 13:14:38,962::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3141::DEBUG::2012-12-09 13:14:38,962::task::978::TaskManager.Task::(_decref) Task=`1c860795-9ef2-472f-928b-054736870ccc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:39,295::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:39,338::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179254 s, 57.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:41,345::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:41,385::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168047 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:43,392::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:43,454::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0292183 s, 35.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:44,334::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:44,343::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000465165 s, 8.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:45,461::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:45,505::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173335 s, 59.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:14:46,476::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:14:46,477::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:46,478::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:46,586::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:46,588::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:46,588::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:46,588::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:14:47,513::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:47,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175386 s, 58.4 MB/s\n'; <rc> = 0
>Thread-3146::DEBUG::2012-12-09 13:14:48,560::task::588::TaskManager.Task::(_updateState) Task=`d4810f13-9053-4a89-ac5b-93f01488cd57`::moving from state init -> state preparing
>Thread-3146::INFO::2012-12-09 13:14:48,561::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3146::INFO::2012-12-09 13:14:48,561::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00902700424194', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-3146::DEBUG::2012-12-09 13:14:48,561::task::1172::TaskManager.Task::(prepare) Task=`d4810f13-9053-4a89-ac5b-93f01488cd57`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00902700424194', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.0', 'code': 358, 'valid': False}}
>Thread-3146::DEBUG::2012-12-09 13:14:48,562::task::588::TaskManager.Task::(_updateState) Task=`d4810f13-9053-4a89-ac5b-93f01488cd57`::moving from state preparing -> state finished
>Thread-3146::DEBUG::2012-12-09 13:14:48,562::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3146::DEBUG::2012-12-09 13:14:48,562::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3146::DEBUG::2012-12-09 13:14:48,563::task::978::TaskManager.Task::(_decref) Task=`d4810f13-9053-4a89-ac5b-93f01488cd57`::ref 0 aborting False
>Thread-3148::DEBUG::2012-12-09 13:14:49,395::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3148::DEBUG::2012-12-09 13:14:49,396::task::588::TaskManager.Task::(_updateState) Task=`29553a56-1883-447b-9c57-d0ef517191df`::moving from state init -> state preparing
>Thread-3148::INFO::2012-12-09 13:14:49,396::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3148::INFO::2012-12-09 13:14:49,397::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3148::DEBUG::2012-12-09 13:14:49,397::task::1172::TaskManager.Task::(prepare) Task=`29553a56-1883-447b-9c57-d0ef517191df`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3148::DEBUG::2012-12-09 13:14:49,397::task::588::TaskManager.Task::(_updateState) Task=`29553a56-1883-447b-9c57-d0ef517191df`::moving from state preparing -> state finished
>Thread-3148::DEBUG::2012-12-09 13:14:49,397::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3148::DEBUG::2012-12-09 13:14:49,397::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3148::DEBUG::2012-12-09 13:14:49,398::task::978::TaskManager.Task::(_decref) Task=`29553a56-1883-447b-9c57-d0ef517191df`::ref 0 aborting False
>Thread-3149::DEBUG::2012-12-09 13:14:49,413::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3149::DEBUG::2012-12-09 13:14:49,414::task::588::TaskManager.Task::(_updateState) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::moving from state init -> state preparing
>Thread-3149::INFO::2012-12-09 13:14:49,414::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3149::DEBUG::2012-12-09 13:14:49,415::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1e28fb4f-1beb-46bf-9338-59d59cf496ea`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3149::DEBUG::2012-12-09 13:14:49,415::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3149::DEBUG::2012-12-09 13:14:49,416::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3149::DEBUG::2012-12-09 13:14:49,416::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1e28fb4f-1beb-46bf-9338-59d59cf496ea`::Granted request
>Thread-3149::DEBUG::2012-12-09 13:14:49,418::task::817::TaskManager.Task::(resourceAcquired) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3149::DEBUG::2012-12-09 13:14:49,418::task::978::TaskManager.Task::(_decref) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::ref 1 aborting False
>Thread-3149::DEBUG::2012-12-09 13:14:49,419::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3149::DEBUG::2012-12-09 13:14:49,420::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3149::DEBUG::2012-12-09 13:14:49,525::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3149::WARNING::2012-12-09 13:14:49,526::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3149::DEBUG::2012-12-09 13:14:49,526::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3149::WARNING::2012-12-09 13:14:49,527::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3149::INFO::2012-12-09 13:14:49,527::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3149::DEBUG::2012-12-09 13:14:49,527::task::1172::TaskManager.Task::(prepare) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3149::DEBUG::2012-12-09 13:14:49,528::task::588::TaskManager.Task::(_updateState) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::moving from state preparing -> state finished
>Thread-3149::DEBUG::2012-12-09 13:14:49,528::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3149::DEBUG::2012-12-09 13:14:49,528::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3149::DEBUG::2012-12-09 13:14:49,528::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3149::DEBUG::2012-12-09 13:14:49,529::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3149::DEBUG::2012-12-09 13:14:49,529::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3149::DEBUG::2012-12-09 13:14:49,529::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3149::DEBUG::2012-12-09 13:14:49,530::task::978::TaskManager.Task::(_decref) Task=`730daf6e-e757-478d-a971-5f4cb13fc52d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:49,562::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:49,632::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300343 s, 34.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:51,641::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:51,682::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163863 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:53,688::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:53,731::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182572 s, 56.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:14:54,346::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:14:54,357::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000451898 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:14:55,739::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:55,796::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0300156 s, 34.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:14:56,589::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:14:56,590::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:14:56,695::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:14:56,697::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:14:56,697::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:14:56,698::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:14:57,803::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:57,863::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02529 s, 40.5 MB/s\n'; <rc> = 0
>Thread-3154::DEBUG::2012-12-09 13:14:59,081::task::588::TaskManager.Task::(_updateState) Task=`763241d9-4e40-49ee-a3c0-f46a5a0be951`::moving from state init -> state preparing
>Thread-3154::INFO::2012-12-09 13:14:59,082::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3154::INFO::2012-12-09 13:14:59,082::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114719867706', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-3154::DEBUG::2012-12-09 13:14:59,082::task::1172::TaskManager.Task::(prepare) Task=`763241d9-4e40-49ee-a3c0-f46a5a0be951`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114719867706', 'lastCheck': '4.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.4', 'code': 358, 'valid': False}}
>Thread-3154::DEBUG::2012-12-09 13:14:59,083::task::588::TaskManager.Task::(_updateState) Task=`763241d9-4e40-49ee-a3c0-f46a5a0be951`::moving from state preparing -> state finished
>Thread-3154::DEBUG::2012-12-09 13:14:59,083::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3154::DEBUG::2012-12-09 13:14:59,083::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3154::DEBUG::2012-12-09 13:14:59,084::task::978::TaskManager.Task::(_decref) Task=`763241d9-4e40-49ee-a3c0-f46a5a0be951`::ref 0 aborting False
>Thread-3156::DEBUG::2012-12-09 13:14:59,684::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3156::DEBUG::2012-12-09 13:14:59,685::task::588::TaskManager.Task::(_updateState) Task=`5eb39947-cf94-4166-890e-6e44a5c345c8`::moving from state init -> state preparing
>Thread-3156::INFO::2012-12-09 13:14:59,686::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3156::INFO::2012-12-09 13:14:59,686::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3156::DEBUG::2012-12-09 13:14:59,686::task::1172::TaskManager.Task::(prepare) Task=`5eb39947-cf94-4166-890e-6e44a5c345c8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3156::DEBUG::2012-12-09 13:14:59,687::task::588::TaskManager.Task::(_updateState) Task=`5eb39947-cf94-4166-890e-6e44a5c345c8`::moving from state preparing -> state finished
>Thread-3156::DEBUG::2012-12-09 13:14:59,687::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3156::DEBUG::2012-12-09 13:14:59,687::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3156::DEBUG::2012-12-09 13:14:59,688::task::978::TaskManager.Task::(_decref) Task=`5eb39947-cf94-4166-890e-6e44a5c345c8`::ref 0 aborting False
>Thread-3157::DEBUG::2012-12-09 13:14:59,702::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3157::DEBUG::2012-12-09 13:14:59,703::task::588::TaskManager.Task::(_updateState) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::moving from state init -> state preparing
>Thread-3157::INFO::2012-12-09 13:14:59,703::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3157::DEBUG::2012-12-09 13:14:59,704::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`efacb874-ebe1-4b40-948d-0a8b899b5e4c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3157::DEBUG::2012-12-09 13:14:59,704::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3157::DEBUG::2012-12-09 13:14:59,705::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3157::DEBUG::2012-12-09 13:14:59,705::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`efacb874-ebe1-4b40-948d-0a8b899b5e4c`::Granted request
>Thread-3157::DEBUG::2012-12-09 13:14:59,705::task::817::TaskManager.Task::(resourceAcquired) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3157::DEBUG::2012-12-09 13:14:59,706::task::978::TaskManager.Task::(_decref) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::ref 1 aborting False
>Thread-3157::DEBUG::2012-12-09 13:14:59,707::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3157::DEBUG::2012-12-09 13:14:59,707::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3157::DEBUG::2012-12-09 13:14:59,836::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3157::WARNING::2012-12-09 13:14:59,837::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3157::DEBUG::2012-12-09 13:14:59,837::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3157::WARNING::2012-12-09 13:14:59,838::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3157::INFO::2012-12-09 13:14:59,838::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3157::DEBUG::2012-12-09 13:14:59,838::task::1172::TaskManager.Task::(prepare) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3157::DEBUG::2012-12-09 13:14:59,838::task::588::TaskManager.Task::(_updateState) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::moving from state preparing -> state finished
>Thread-3157::DEBUG::2012-12-09 13:14:59,839::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3157::DEBUG::2012-12-09 13:14:59,839::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3157::DEBUG::2012-12-09 13:14:59,839::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3157::DEBUG::2012-12-09 13:14:59,839::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3157::DEBUG::2012-12-09 13:14:59,840::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3157::DEBUG::2012-12-09 13:14:59,840::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3157::DEBUG::2012-12-09 13:14:59,840::task::978::TaskManager.Task::(_decref) Task=`ece9a083-bbb9-47db-ae69-dffd0f6322c1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:14:59,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:14:59,917::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164615 s, 62.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:01,924::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:01,965::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170546 s, 60.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:03,972::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:04,017::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220836 s, 46.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:15:04,360::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:04,368::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000537044 s, 7.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:06,024::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:06,067::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193943 s, 52.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:06,699::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:06,699::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:15:06,799::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:06,801::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:06,801::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:06,801::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:15:08,074::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:08,128::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244979 s, 41.8 MB/s\n'; <rc> = 0
>Thread-3162::DEBUG::2012-12-09 13:15:09,666::task::588::TaskManager.Task::(_updateState) Task=`5f9c7b7f-67b3-4131-aa24-051d3ad426ce`::moving from state init -> state preparing
>Thread-3162::INFO::2012-12-09 13:15:09,667::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3162::INFO::2012-12-09 13:15:09,667::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00805401802063', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-3162::DEBUG::2012-12-09 13:15:09,667::task::1172::TaskManager.Task::(prepare) Task=`5f9c7b7f-67b3-4131-aa24-051d3ad426ce`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00805401802063', 'lastCheck': '5.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.9', 'code': 358, 'valid': False}}
>Thread-3162::DEBUG::2012-12-09 13:15:09,667::task::588::TaskManager.Task::(_updateState) Task=`5f9c7b7f-67b3-4131-aa24-051d3ad426ce`::moving from state preparing -> state finished
>Thread-3162::DEBUG::2012-12-09 13:15:09,667::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3162::DEBUG::2012-12-09 13:15:09,668::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3162::DEBUG::2012-12-09 13:15:09,668::task::978::TaskManager.Task::(_decref) Task=`5f9c7b7f-67b3-4131-aa24-051d3ad426ce`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:15:10,134::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:10,172::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015334 s, 66.8 MB/s\n'; <rc> = 0
>Thread-3164::DEBUG::2012-12-09 13:15:10,474::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3164::DEBUG::2012-12-09 13:15:10,474::task::588::TaskManager.Task::(_updateState) Task=`95a58876-63b2-4283-851a-9fd3f30337ec`::moving from state init -> state preparing
>Thread-3164::INFO::2012-12-09 13:15:10,474::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3164::INFO::2012-12-09 13:15:10,475::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3164::DEBUG::2012-12-09 13:15:10,475::task::1172::TaskManager.Task::(prepare) Task=`95a58876-63b2-4283-851a-9fd3f30337ec`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3164::DEBUG::2012-12-09 13:15:10,475::task::588::TaskManager.Task::(_updateState) Task=`95a58876-63b2-4283-851a-9fd3f30337ec`::moving from state preparing -> state finished
>Thread-3164::DEBUG::2012-12-09 13:15:10,475::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3164::DEBUG::2012-12-09 13:15:10,475::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3164::DEBUG::2012-12-09 13:15:10,476::task::978::TaskManager.Task::(_decref) Task=`95a58876-63b2-4283-851a-9fd3f30337ec`::ref 0 aborting False
>Thread-3165::DEBUG::2012-12-09 13:15:10,493::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3165::DEBUG::2012-12-09 13:15:10,494::task::588::TaskManager.Task::(_updateState) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::moving from state init -> state preparing
>Thread-3165::INFO::2012-12-09 13:15:10,494::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3165::DEBUG::2012-12-09 13:15:10,495::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67e3a181-92db-4196-86ee-b4ce42b87391`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3165::DEBUG::2012-12-09 13:15:10,495::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3165::DEBUG::2012-12-09 13:15:10,496::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3165::DEBUG::2012-12-09 13:15:10,496::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`67e3a181-92db-4196-86ee-b4ce42b87391`::Granted request
>Thread-3165::DEBUG::2012-12-09 13:15:10,497::task::817::TaskManager.Task::(resourceAcquired) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3165::DEBUG::2012-12-09 13:15:10,497::task::978::TaskManager.Task::(_decref) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::ref 1 aborting False
>Thread-3165::DEBUG::2012-12-09 13:15:10,499::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3165::DEBUG::2012-12-09 13:15:10,500::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3165::DEBUG::2012-12-09 13:15:10,616::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3165::WARNING::2012-12-09 13:15:10,618::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3165::DEBUG::2012-12-09 13:15:10,618::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3165::WARNING::2012-12-09 13:15:10,619::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3165::INFO::2012-12-09 13:15:10,620::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3165::DEBUG::2012-12-09 13:15:10,620::task::1172::TaskManager.Task::(prepare) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3165::DEBUG::2012-12-09 13:15:10,620::task::588::TaskManager.Task::(_updateState) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::moving from state preparing -> state finished
>Thread-3165::DEBUG::2012-12-09 13:15:10,621::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3165::DEBUG::2012-12-09 13:15:10,621::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3165::DEBUG::2012-12-09 13:15:10,621::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3165::DEBUG::2012-12-09 13:15:10,621::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3165::DEBUG::2012-12-09 13:15:10,622::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3165::DEBUG::2012-12-09 13:15:10,622::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3165::DEBUG::2012-12-09 13:15:10,622::task::978::TaskManager.Task::(_decref) Task=`efa896f6-cadc-4042-85b6-497b430b61ef`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:15:12,179::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:12,224::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218765 s, 46.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:14,231::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:14,273::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176333 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:15:14,371::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:14,393::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.0011432 s, 3.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:16,279::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:16,320::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174752 s, 58.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:16,802::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:16,803::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:15:16,896::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:16,897::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:16,897::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:16,898::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:15:18,326::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:18,366::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172477 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:20,373::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:20,412::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167794 s, 61.0 MB/s\n'; <rc> = 0
>Thread-3170::DEBUG::2012-12-09 13:15:20,467::task::588::TaskManager.Task::(_updateState) Task=`a54cff1d-b300-41fe-af25-d35d0bbfca94`::moving from state init -> state preparing
>Thread-3170::INFO::2012-12-09 13:15:20,467::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3170::INFO::2012-12-09 13:15:20,468::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0224761962891', 'lastCheck': '6.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-3170::DEBUG::2012-12-09 13:15:20,468::task::1172::TaskManager.Task::(prepare) Task=`a54cff1d-b300-41fe-af25-d35d0bbfca94`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0224761962891', 'lastCheck': '6.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.6', 'code': 358, 'valid': False}}
>Thread-3170::DEBUG::2012-12-09 13:15:20,468::task::588::TaskManager.Task::(_updateState) Task=`a54cff1d-b300-41fe-af25-d35d0bbfca94`::moving from state preparing -> state finished
>Thread-3170::DEBUG::2012-12-09 13:15:20,468::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3170::DEBUG::2012-12-09 13:15:20,469::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3170::DEBUG::2012-12-09 13:15:20,469::task::978::TaskManager.Task::(_decref) Task=`a54cff1d-b300-41fe-af25-d35d0bbfca94`::ref 0 aborting False
>Thread-3172::DEBUG::2012-12-09 13:15:21,141::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3172::DEBUG::2012-12-09 13:15:21,141::task::588::TaskManager.Task::(_updateState) Task=`2d3e47e1-629c-46b0-b4cf-838b7a2b12b9`::moving from state init -> state preparing
>Thread-3172::INFO::2012-12-09 13:15:21,141::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3172::INFO::2012-12-09 13:15:21,142::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3172::DEBUG::2012-12-09 13:15:21,142::task::1172::TaskManager.Task::(prepare) Task=`2d3e47e1-629c-46b0-b4cf-838b7a2b12b9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3172::DEBUG::2012-12-09 13:15:21,142::task::588::TaskManager.Task::(_updateState) Task=`2d3e47e1-629c-46b0-b4cf-838b7a2b12b9`::moving from state preparing -> state finished
>Thread-3172::DEBUG::2012-12-09 13:15:21,142::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3172::DEBUG::2012-12-09 13:15:21,143::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3172::DEBUG::2012-12-09 13:15:21,143::task::978::TaskManager.Task::(_decref) Task=`2d3e47e1-629c-46b0-b4cf-838b7a2b12b9`::ref 0 aborting False
>Thread-3173::DEBUG::2012-12-09 13:15:21,154::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3173::DEBUG::2012-12-09 13:15:21,155::task::588::TaskManager.Task::(_updateState) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::moving from state init -> state preparing
>Thread-3173::INFO::2012-12-09 13:15:21,155::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3173::DEBUG::2012-12-09 13:15:21,156::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7882cc95-a158-41ba-9e18-b970f7bd0685`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3173::DEBUG::2012-12-09 13:15:21,156::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3173::DEBUG::2012-12-09 13:15:21,157::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3173::DEBUG::2012-12-09 13:15:21,157::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7882cc95-a158-41ba-9e18-b970f7bd0685`::Granted request
>Thread-3173::DEBUG::2012-12-09 13:15:21,157::task::817::TaskManager.Task::(resourceAcquired) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3173::DEBUG::2012-12-09 13:15:21,158::task::978::TaskManager.Task::(_decref) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::ref 1 aborting False
>Thread-3173::DEBUG::2012-12-09 13:15:21,159::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3173::DEBUG::2012-12-09 13:15:21,159::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3173::DEBUG::2012-12-09 13:15:21,262::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3173::WARNING::2012-12-09 13:15:21,263::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3173::DEBUG::2012-12-09 13:15:21,263::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3173::WARNING::2012-12-09 13:15:21,264::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3173::INFO::2012-12-09 13:15:21,264::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3173::DEBUG::2012-12-09 13:15:21,264::task::1172::TaskManager.Task::(prepare) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3173::DEBUG::2012-12-09 13:15:21,265::task::588::TaskManager.Task::(_updateState) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::moving from state preparing -> state finished
>Thread-3173::DEBUG::2012-12-09 13:15:21,265::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3173::DEBUG::2012-12-09 13:15:21,265::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3173::DEBUG::2012-12-09 13:15:21,265::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3173::DEBUG::2012-12-09 13:15:21,265::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3173::DEBUG::2012-12-09 13:15:21,266::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3173::DEBUG::2012-12-09 13:15:21,266::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3173::DEBUG::2012-12-09 13:15:21,266::task::978::TaskManager.Task::(_decref) Task=`66010e4b-0356-4d43-b9e8-c10dc8f22082`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:15:22,419::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:22,457::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159385 s, 64.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:15:24,396::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:24,406::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00040159 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:24,463::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:24,503::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170161 s, 60.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:26,510::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:26,558::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227222 s, 45.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:26,899::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:26,901::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:15:26,998::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:26,999::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:26,999::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:26,999::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:15:28,565::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:28,607::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173873 s, 58.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:30,615::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:30,682::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0288141 s, 35.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:32,689::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:32,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211898 s, 48.3 MB/s\n'; <rc> = 0
>Thread-3178::DEBUG::2012-12-09 13:15:34,154::task::588::TaskManager.Task::(_updateState) Task=`fb12c772-1f58-461d-be23-74db21427cd3`::moving from state init -> state preparing
>Thread-3178::INFO::2012-12-09 13:15:34,155::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3178::INFO::2012-12-09 13:15:34,155::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102870464325', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.2', 'code': 358, 'valid': False}}
>Thread-3178::DEBUG::2012-12-09 13:15:34,155::task::1172::TaskManager.Task::(prepare) Task=`fb12c772-1f58-461d-be23-74db21427cd3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102870464325', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.2', 'code': 358, 'valid': False}}
>Thread-3178::DEBUG::2012-12-09 13:15:34,155::task::588::TaskManager.Task::(_updateState) Task=`fb12c772-1f58-461d-be23-74db21427cd3`::moving from state preparing -> state finished
>Thread-3178::DEBUG::2012-12-09 13:15:34,155::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3178::DEBUG::2012-12-09 13:15:34,156::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3178::DEBUG::2012-12-09 13:15:34,156::task::978::TaskManager.Task::(_decref) Task=`fb12c772-1f58-461d-be23-74db21427cd3`::ref 0 aborting False
>Thread-3180::DEBUG::2012-12-09 13:15:34,270::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3180::DEBUG::2012-12-09 13:15:34,271::task::588::TaskManager.Task::(_updateState) Task=`3daa4b19-d835-418c-aaaa-23851497d2d9`::moving from state init -> state preparing
>Thread-3180::INFO::2012-12-09 13:15:34,271::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3180::INFO::2012-12-09 13:15:34,272::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3180::DEBUG::2012-12-09 13:15:34,272::task::1172::TaskManager.Task::(prepare) Task=`3daa4b19-d835-418c-aaaa-23851497d2d9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3180::DEBUG::2012-12-09 13:15:34,272::task::588::TaskManager.Task::(_updateState) Task=`3daa4b19-d835-418c-aaaa-23851497d2d9`::moving from state preparing -> state finished
>Thread-3180::DEBUG::2012-12-09 13:15:34,272::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3180::DEBUG::2012-12-09 13:15:34,272::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3180::DEBUG::2012-12-09 13:15:34,273::task::978::TaskManager.Task::(_decref) Task=`3daa4b19-d835-418c-aaaa-23851497d2d9`::ref 0 aborting False
>Thread-3181::DEBUG::2012-12-09 13:15:34,285::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3181::DEBUG::2012-12-09 13:15:34,285::task::588::TaskManager.Task::(_updateState) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::moving from state init -> state preparing
>Thread-3181::INFO::2012-12-09 13:15:34,285::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3181::DEBUG::2012-12-09 13:15:34,286::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f186ae25-f839-49ea-889e-78f60a563aa1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3181::DEBUG::2012-12-09 13:15:34,286::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3181::DEBUG::2012-12-09 13:15:34,286::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3181::DEBUG::2012-12-09 13:15:34,286::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f186ae25-f839-49ea-889e-78f60a563aa1`::Granted request
>Thread-3181::DEBUG::2012-12-09 13:15:34,287::task::817::TaskManager.Task::(resourceAcquired) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3181::DEBUG::2012-12-09 13:15:34,287::task::978::TaskManager.Task::(_decref) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::ref 1 aborting False
>Thread-3181::DEBUG::2012-12-09 13:15:34,288::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3181::DEBUG::2012-12-09 13:15:34,288::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3181::DEBUG::2012-12-09 13:15:34,385::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3181::WARNING::2012-12-09 13:15:34,386::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3181::DEBUG::2012-12-09 13:15:34,386::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3181::WARNING::2012-12-09 13:15:34,387::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3181::INFO::2012-12-09 13:15:34,387::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3181::DEBUG::2012-12-09 13:15:34,387::task::1172::TaskManager.Task::(prepare) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3181::DEBUG::2012-12-09 13:15:34,387::task::588::TaskManager.Task::(_updateState) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::moving from state preparing -> state finished
>Thread-3181::DEBUG::2012-12-09 13:15:34,388::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3181::DEBUG::2012-12-09 13:15:34,388::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3181::DEBUG::2012-12-09 13:15:34,388::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3181::DEBUG::2012-12-09 13:15:34,388::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3181::DEBUG::2012-12-09 13:15:34,389::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3181::DEBUG::2012-12-09 13:15:34,389::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3181::DEBUG::2012-12-09 13:15:34,389::task::978::TaskManager.Task::(_decref) Task=`9cd6985d-7e13-43ff-898e-ce77e1b157c9`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:15:34,409::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:34,422::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000702475 s, 5.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:34,740::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:34,780::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164746 s, 62.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:36,786::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:36,825::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165942 s, 61.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:37,000::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:37,001::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:15:37,111::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:37,112::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:37,112::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:37,113::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:15:38,832::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:38,872::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016194 s, 63.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:40,879::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:40,918::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016181 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:42,925::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:42,968::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176183 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:15:44,428::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:44,441::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000757985 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:44,975::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:45,016::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017793 s, 57.6 MB/s\n'; <rc> = 0
>Thread-3186::DEBUG::2012-12-09 13:15:45,887::task::588::TaskManager.Task::(_updateState) Task=`9436503a-ccca-4895-a3c9-891de55c25e3`::moving from state init -> state preparing
>Thread-3186::INFO::2012-12-09 13:15:45,887::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3186::INFO::2012-12-09 13:15:45,888::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136229991913', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-3186::DEBUG::2012-12-09 13:15:45,888::task::1172::TaskManager.Task::(prepare) Task=`9436503a-ccca-4895-a3c9-891de55c25e3`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136229991913', 'lastCheck': '1.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.8', 'code': 358, 'valid': False}}
>Thread-3186::DEBUG::2012-12-09 13:15:45,888::task::588::TaskManager.Task::(_updateState) Task=`9436503a-ccca-4895-a3c9-891de55c25e3`::moving from state preparing -> state finished
>Thread-3186::DEBUG::2012-12-09 13:15:45,888::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3186::DEBUG::2012-12-09 13:15:45,888::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3186::DEBUG::2012-12-09 13:15:45,889::task::978::TaskManager.Task::(_decref) Task=`9436503a-ccca-4895-a3c9-891de55c25e3`::ref 0 aborting False
>Thread-3188::DEBUG::2012-12-09 13:15:46,190::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3188::DEBUG::2012-12-09 13:15:46,191::task::588::TaskManager.Task::(_updateState) Task=`1f7d5642-072d-48b2-bbcc-1b159e850c52`::moving from state init -> state preparing
>Thread-3188::INFO::2012-12-09 13:15:46,191::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3188::INFO::2012-12-09 13:15:46,191::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3188::DEBUG::2012-12-09 13:15:46,192::task::1172::TaskManager.Task::(prepare) Task=`1f7d5642-072d-48b2-bbcc-1b159e850c52`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3188::DEBUG::2012-12-09 13:15:46,192::task::588::TaskManager.Task::(_updateState) Task=`1f7d5642-072d-48b2-bbcc-1b159e850c52`::moving from state preparing -> state finished
>Thread-3188::DEBUG::2012-12-09 13:15:46,192::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3188::DEBUG::2012-12-09 13:15:46,192::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3188::DEBUG::2012-12-09 13:15:46,192::task::978::TaskManager.Task::(_decref) Task=`1f7d5642-072d-48b2-bbcc-1b159e850c52`::ref 0 aborting False
>Thread-3189::DEBUG::2012-12-09 13:15:46,204::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3189::DEBUG::2012-12-09 13:15:46,204::task::588::TaskManager.Task::(_updateState) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::moving from state init -> state preparing
>Thread-3189::INFO::2012-12-09 13:15:46,204::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3189::DEBUG::2012-12-09 13:15:46,205::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b07f8a3d-184a-4fc3-9d27-bc706e7fac5a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3189::DEBUG::2012-12-09 13:15:46,205::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3189::DEBUG::2012-12-09 13:15:46,205::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3189::DEBUG::2012-12-09 13:15:46,205::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b07f8a3d-184a-4fc3-9d27-bc706e7fac5a`::Granted request
>Thread-3189::DEBUG::2012-12-09 13:15:46,206::task::817::TaskManager.Task::(resourceAcquired) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3189::DEBUG::2012-12-09 13:15:46,206::task::978::TaskManager.Task::(_decref) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::ref 1 aborting False
>Thread-3189::DEBUG::2012-12-09 13:15:46,207::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3189::DEBUG::2012-12-09 13:15:46,207::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3189::DEBUG::2012-12-09 13:15:46,305::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3189::WARNING::2012-12-09 13:15:46,306::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3189::DEBUG::2012-12-09 13:15:46,307::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3189::WARNING::2012-12-09 13:15:46,307::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3189::INFO::2012-12-09 13:15:46,307::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3189::DEBUG::2012-12-09 13:15:46,308::task::1172::TaskManager.Task::(prepare) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3189::DEBUG::2012-12-09 13:15:46,308::task::588::TaskManager.Task::(_updateState) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::moving from state preparing -> state finished
>Thread-3189::DEBUG::2012-12-09 13:15:46,308::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3189::DEBUG::2012-12-09 13:15:46,308::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3189::DEBUG::2012-12-09 13:15:46,308::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3189::DEBUG::2012-12-09 13:15:46,309::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3189::DEBUG::2012-12-09 13:15:46,309::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3189::DEBUG::2012-12-09 13:15:46,309::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3189::DEBUG::2012-12-09 13:15:46,309::task::978::TaskManager.Task::(_decref) Task=`6c6852ab-9680-4754-a520-72fb47aac0dc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:15:47,022::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:47,060::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151654 s, 67.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:47,114::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:47,115::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:15:47,214::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:47,215::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:47,216::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:47,217::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:15:49,066::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:49,116::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206371 s, 49.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:51,123::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:51,164::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165608 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:53,170::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:53,219::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219522 s, 46.6 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:15:54,444::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:15:54,455::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000373182 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:15:55,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:55,265::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161356 s, 63.5 MB/s\n'; <rc> = 0
>Thread-3194::DEBUG::2012-12-09 13:15:57,194::task::588::TaskManager.Task::(_updateState) Task=`34b25e7e-b76e-421e-ac29-a22a7f70e0be`::moving from state init -> state preparing
>Thread-3194::INFO::2012-12-09 13:15:57,195::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3194::INFO::2012-12-09 13:15:57,195::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0111310482025', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-3194::DEBUG::2012-12-09 13:15:57,195::task::1172::TaskManager.Task::(prepare) Task=`34b25e7e-b76e-421e-ac29-a22a7f70e0be`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0111310482025', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-3194::DEBUG::2012-12-09 13:15:57,195::task::588::TaskManager.Task::(_updateState) Task=`34b25e7e-b76e-421e-ac29-a22a7f70e0be`::moving from state preparing -> state finished
>Thread-3194::DEBUG::2012-12-09 13:15:57,196::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3194::DEBUG::2012-12-09 13:15:57,196::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3194::DEBUG::2012-12-09 13:15:57,196::task::978::TaskManager.Task::(_decref) Task=`34b25e7e-b76e-421e-ac29-a22a7f70e0be`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:15:57,217::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:15:57,218::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:57,272::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:57,311::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162539 s, 63.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:15:57,320::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:15:57,321::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:15:57,321::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:15:57,322::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3196::DEBUG::2012-12-09 13:15:57,501::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3196::DEBUG::2012-12-09 13:15:57,502::task::588::TaskManager.Task::(_updateState) Task=`c7e06068-b9ca-48d6-9283-3a675abf775b`::moving from state init -> state preparing
>Thread-3196::INFO::2012-12-09 13:15:57,502::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3196::INFO::2012-12-09 13:15:57,503::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3196::DEBUG::2012-12-09 13:15:57,503::task::1172::TaskManager.Task::(prepare) Task=`c7e06068-b9ca-48d6-9283-3a675abf775b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3196::DEBUG::2012-12-09 13:15:57,503::task::588::TaskManager.Task::(_updateState) Task=`c7e06068-b9ca-48d6-9283-3a675abf775b`::moving from state preparing -> state finished
>Thread-3196::DEBUG::2012-12-09 13:15:57,503::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3196::DEBUG::2012-12-09 13:15:57,503::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3196::DEBUG::2012-12-09 13:15:57,504::task::978::TaskManager.Task::(_decref) Task=`c7e06068-b9ca-48d6-9283-3a675abf775b`::ref 0 aborting False
>Thread-3197::DEBUG::2012-12-09 13:15:57,517::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3197::DEBUG::2012-12-09 13:15:57,517::task::588::TaskManager.Task::(_updateState) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::moving from state init -> state preparing
>Thread-3197::INFO::2012-12-09 13:15:57,517::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3197::DEBUG::2012-12-09 13:15:57,518::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6182121c-6713-4776-bbcf-8fb0cd604e39`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3197::DEBUG::2012-12-09 13:15:57,518::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3197::DEBUG::2012-12-09 13:15:57,518::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3197::DEBUG::2012-12-09 13:15:57,518::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6182121c-6713-4776-bbcf-8fb0cd604e39`::Granted request
>Thread-3197::DEBUG::2012-12-09 13:15:57,519::task::817::TaskManager.Task::(resourceAcquired) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3197::DEBUG::2012-12-09 13:15:57,519::task::978::TaskManager.Task::(_decref) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::ref 1 aborting False
>Thread-3197::DEBUG::2012-12-09 13:15:57,520::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3197::DEBUG::2012-12-09 13:15:57,520::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3197::DEBUG::2012-12-09 13:15:57,622::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3197::WARNING::2012-12-09 13:15:57,623::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3197::DEBUG::2012-12-09 13:15:57,623::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3197::WARNING::2012-12-09 13:15:57,624::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3197::INFO::2012-12-09 13:15:57,624::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3197::DEBUG::2012-12-09 13:15:57,624::task::1172::TaskManager.Task::(prepare) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3197::DEBUG::2012-12-09 13:15:57,625::task::588::TaskManager.Task::(_updateState) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::moving from state preparing -> state finished
>Thread-3197::DEBUG::2012-12-09 13:15:57,625::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3197::DEBUG::2012-12-09 13:15:57,625::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3197::DEBUG::2012-12-09 13:15:57,625::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3197::DEBUG::2012-12-09 13:15:57,626::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3197::DEBUG::2012-12-09 13:15:57,626::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3197::DEBUG::2012-12-09 13:15:57,626::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3197::DEBUG::2012-12-09 13:15:57,627::task::978::TaskManager.Task::(_decref) Task=`bbb821dc-6413-42ca-9d8c-a52a15e34b53`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:15:59,319::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:15:59,361::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016432 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:01,368::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:01,411::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190677 s, 53.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:03,418::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:03,466::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0238452 s, 42.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:16:04,458::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:04,473::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000621772 s, 6.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:05,473::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:05,513::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168958 s, 60.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:07,323::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:07,323::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:07,423::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:07,424::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:07,425::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:07,425::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:07,520::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:07,564::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193254 s, 53.0 MB/s\n'; <rc> = 0
>Thread-3202::DEBUG::2012-12-09 13:16:08,922::task::588::TaskManager.Task::(_updateState) Task=`aad7f1bb-4a23-4d64-b236-9f4f2d52b5ec`::moving from state init -> state preparing
>Thread-3202::INFO::2012-12-09 13:16:08,923::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3202::INFO::2012-12-09 13:16:08,923::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154869556427', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-3202::DEBUG::2012-12-09 13:16:08,923::task::1172::TaskManager.Task::(prepare) Task=`aad7f1bb-4a23-4d64-b236-9f4f2d52b5ec`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154869556427', 'lastCheck': '4.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.5', 'code': 358, 'valid': False}}
>Thread-3202::DEBUG::2012-12-09 13:16:08,923::task::588::TaskManager.Task::(_updateState) Task=`aad7f1bb-4a23-4d64-b236-9f4f2d52b5ec`::moving from state preparing -> state finished
>Thread-3202::DEBUG::2012-12-09 13:16:08,923::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3202::DEBUG::2012-12-09 13:16:08,924::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3202::DEBUG::2012-12-09 13:16:08,924::task::978::TaskManager.Task::(_decref) Task=`aad7f1bb-4a23-4d64-b236-9f4f2d52b5ec`::ref 0 aborting False
>Thread-3204::DEBUG::2012-12-09 13:16:09,218::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3204::DEBUG::2012-12-09 13:16:09,219::task::588::TaskManager.Task::(_updateState) Task=`bd5dc12e-f446-44c4-9e47-30bb2a643382`::moving from state init -> state preparing
>Thread-3204::INFO::2012-12-09 13:16:09,219::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3204::INFO::2012-12-09 13:16:09,219::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3204::DEBUG::2012-12-09 13:16:09,220::task::1172::TaskManager.Task::(prepare) Task=`bd5dc12e-f446-44c4-9e47-30bb2a643382`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3204::DEBUG::2012-12-09 13:16:09,220::task::588::TaskManager.Task::(_updateState) Task=`bd5dc12e-f446-44c4-9e47-30bb2a643382`::moving from state preparing -> state finished
>Thread-3204::DEBUG::2012-12-09 13:16:09,220::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3204::DEBUG::2012-12-09 13:16:09,220::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3204::DEBUG::2012-12-09 13:16:09,221::task::978::TaskManager.Task::(_decref) Task=`bd5dc12e-f446-44c4-9e47-30bb2a643382`::ref 0 aborting False
>Thread-3205::DEBUG::2012-12-09 13:16:09,237::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3205::DEBUG::2012-12-09 13:16:09,238::task::588::TaskManager.Task::(_updateState) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::moving from state init -> state preparing
>Thread-3205::INFO::2012-12-09 13:16:09,238::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3205::DEBUG::2012-12-09 13:16:09,238::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`853ea089-15ec-4876-ad19-e7082ac3862b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3205::DEBUG::2012-12-09 13:16:09,239::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3205::DEBUG::2012-12-09 13:16:09,239::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3205::DEBUG::2012-12-09 13:16:09,239::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`853ea089-15ec-4876-ad19-e7082ac3862b`::Granted request
>Thread-3205::DEBUG::2012-12-09 13:16:09,239::task::817::TaskManager.Task::(resourceAcquired) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3205::DEBUG::2012-12-09 13:16:09,239::task::978::TaskManager.Task::(_decref) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::ref 1 aborting False
>Thread-3205::DEBUG::2012-12-09 13:16:09,240::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3205::DEBUG::2012-12-09 13:16:09,241::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3205::DEBUG::2012-12-09 13:16:09,338::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3205::WARNING::2012-12-09 13:16:09,341::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3205::DEBUG::2012-12-09 13:16:09,341::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3205::WARNING::2012-12-09 13:16:09,342::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3205::INFO::2012-12-09 13:16:09,343::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3205::DEBUG::2012-12-09 13:16:09,343::task::1172::TaskManager.Task::(prepare) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3205::DEBUG::2012-12-09 13:16:09,344::task::588::TaskManager.Task::(_updateState) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::moving from state preparing -> state finished
>Thread-3205::DEBUG::2012-12-09 13:16:09,344::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3205::DEBUG::2012-12-09 13:16:09,344::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3205::DEBUG::2012-12-09 13:16:09,345::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3205::DEBUG::2012-12-09 13:16:09,345::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3205::DEBUG::2012-12-09 13:16:09,345::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3205::DEBUG::2012-12-09 13:16:09,345::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3205::DEBUG::2012-12-09 13:16:09,345::task::978::TaskManager.Task::(_decref) Task=`7fb2fb5e-38a2-4e5d-aaa1-c8a283bc7324`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:09,571::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:09,613::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182551 s, 56.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:11,620::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:11,659::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163254 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:13,666::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:13,708::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176209 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:16:14,477::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:14,485::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000499169 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:15,714::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:15,755::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169826 s, 60.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:17,426::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:17,427::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:17,528::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:17,529::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:17,529::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:17,529::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:17,762::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:17,800::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152805 s, 67.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:19,807::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:19,845::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160043 s, 64.0 MB/s\n'; <rc> = 0
>Thread-3210::DEBUG::2012-12-09 13:16:20,081::task::588::TaskManager.Task::(_updateState) Task=`69ca24ed-43e9-477a-8e4b-bdfc12e47254`::moving from state init -> state preparing
>Thread-3210::INFO::2012-12-09 13:16:20,082::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3210::INFO::2012-12-09 13:16:20,082::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00904417037964', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-3210::DEBUG::2012-12-09 13:16:20,083::task::1172::TaskManager.Task::(prepare) Task=`69ca24ed-43e9-477a-8e4b-bdfc12e47254`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00904417037964', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.6', 'code': 358, 'valid': False}}
>Thread-3210::DEBUG::2012-12-09 13:16:20,083::task::588::TaskManager.Task::(_updateState) Task=`69ca24ed-43e9-477a-8e4b-bdfc12e47254`::moving from state preparing -> state finished
>Thread-3210::DEBUG::2012-12-09 13:16:20,084::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3210::DEBUG::2012-12-09 13:16:20,084::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3210::DEBUG::2012-12-09 13:16:20,084::task::978::TaskManager.Task::(_decref) Task=`69ca24ed-43e9-477a-8e4b-bdfc12e47254`::ref 0 aborting False
>Thread-3212::DEBUG::2012-12-09 13:16:20,286::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3212::DEBUG::2012-12-09 13:16:20,286::task::588::TaskManager.Task::(_updateState) Task=`899f4037-d21e-4b53-8812-d2521de3c0aa`::moving from state init -> state preparing
>Thread-3212::INFO::2012-12-09 13:16:20,286::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3212::INFO::2012-12-09 13:16:20,287::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3212::DEBUG::2012-12-09 13:16:20,287::task::1172::TaskManager.Task::(prepare) Task=`899f4037-d21e-4b53-8812-d2521de3c0aa`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3212::DEBUG::2012-12-09 13:16:20,287::task::588::TaskManager.Task::(_updateState) Task=`899f4037-d21e-4b53-8812-d2521de3c0aa`::moving from state preparing -> state finished
>Thread-3212::DEBUG::2012-12-09 13:16:20,287::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3212::DEBUG::2012-12-09 13:16:20,287::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3212::DEBUG::2012-12-09 13:16:20,288::task::978::TaskManager.Task::(_decref) Task=`899f4037-d21e-4b53-8812-d2521de3c0aa`::ref 0 aborting False
>Thread-3213::DEBUG::2012-12-09 13:16:20,300::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3213::DEBUG::2012-12-09 13:16:20,301::task::588::TaskManager.Task::(_updateState) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::moving from state init -> state preparing
>Thread-3213::INFO::2012-12-09 13:16:20,301::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3213::DEBUG::2012-12-09 13:16:20,302::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4aaf29a8-be5a-4501-ade9-b3e49a781b25`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3213::DEBUG::2012-12-09 13:16:20,302::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3213::DEBUG::2012-12-09 13:16:20,303::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3213::DEBUG::2012-12-09 13:16:20,303::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`4aaf29a8-be5a-4501-ade9-b3e49a781b25`::Granted request
>Thread-3213::DEBUG::2012-12-09 13:16:20,304::task::817::TaskManager.Task::(resourceAcquired) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3213::DEBUG::2012-12-09 13:16:20,304::task::978::TaskManager.Task::(_decref) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::ref 1 aborting False
>Thread-3213::DEBUG::2012-12-09 13:16:20,306::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3213::DEBUG::2012-12-09 13:16:20,307::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3213::DEBUG::2012-12-09 13:16:20,413::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3213::WARNING::2012-12-09 13:16:20,415::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3213::DEBUG::2012-12-09 13:16:20,415::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3213::WARNING::2012-12-09 13:16:20,416::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3213::INFO::2012-12-09 13:16:20,416::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3213::DEBUG::2012-12-09 13:16:20,416::task::1172::TaskManager.Task::(prepare) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3213::DEBUG::2012-12-09 13:16:20,417::task::588::TaskManager.Task::(_updateState) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::moving from state preparing -> state finished
>Thread-3213::DEBUG::2012-12-09 13:16:20,417::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3213::DEBUG::2012-12-09 13:16:20,417::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3213::DEBUG::2012-12-09 13:16:20,417::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3213::DEBUG::2012-12-09 13:16:20,418::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3213::DEBUG::2012-12-09 13:16:20,418::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3213::DEBUG::2012-12-09 13:16:20,418::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3213::DEBUG::2012-12-09 13:16:20,418::task::978::TaskManager.Task::(_decref) Task=`f121a0d1-be53-4af5-aa9c-49ab27261a14`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:21,852::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:21,893::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161868 s, 63.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:23,902::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:23,944::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019309 s, 53.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:16:24,489::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:24,504::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000710269 s, 5.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:25,951::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:25,991::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162154 s, 63.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:27,530::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:27,531::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:27,629::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:27,631::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:27,631::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:27,632::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:27,998::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:28,038::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168794 s, 60.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:30,044::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:30,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171291 s, 59.8 MB/s\n'; <rc> = 0
>Thread-3218::DEBUG::2012-12-09 13:16:31,559::task::588::TaskManager.Task::(_updateState) Task=`99cd1642-11e5-4f4f-a176-86487d163d47`::moving from state init -> state preparing
>Thread-3218::INFO::2012-12-09 13:16:31,560::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3218::INFO::2012-12-09 13:16:31,560::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156199932098', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-3218::DEBUG::2012-12-09 13:16:31,560::task::1172::TaskManager.Task::(prepare) Task=`99cd1642-11e5-4f4f-a176-86487d163d47`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0156199932098', 'lastCheck': '7.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.9', 'code': 358, 'valid': False}}
>Thread-3218::DEBUG::2012-12-09 13:16:31,561::task::588::TaskManager.Task::(_updateState) Task=`99cd1642-11e5-4f4f-a176-86487d163d47`::moving from state preparing -> state finished
>Thread-3218::DEBUG::2012-12-09 13:16:31,561::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3218::DEBUG::2012-12-09 13:16:31,561::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3218::DEBUG::2012-12-09 13:16:31,562::task::978::TaskManager.Task::(_decref) Task=`99cd1642-11e5-4f4f-a176-86487d163d47`::ref 0 aborting False
>Thread-3220::DEBUG::2012-12-09 13:16:32,002::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3220::DEBUG::2012-12-09 13:16:32,003::task::588::TaskManager.Task::(_updateState) Task=`15bd2e15-d202-429c-80b4-299b4f70ae5f`::moving from state init -> state preparing
>Thread-3220::INFO::2012-12-09 13:16:32,003::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3220::INFO::2012-12-09 13:16:32,003::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3220::DEBUG::2012-12-09 13:16:32,004::task::1172::TaskManager.Task::(prepare) Task=`15bd2e15-d202-429c-80b4-299b4f70ae5f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3220::DEBUG::2012-12-09 13:16:32,004::task::588::TaskManager.Task::(_updateState) Task=`15bd2e15-d202-429c-80b4-299b4f70ae5f`::moving from state preparing -> state finished
>Thread-3220::DEBUG::2012-12-09 13:16:32,004::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3220::DEBUG::2012-12-09 13:16:32,004::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3220::DEBUG::2012-12-09 13:16:32,004::task::978::TaskManager.Task::(_decref) Task=`15bd2e15-d202-429c-80b4-299b4f70ae5f`::ref 0 aborting False
>Thread-3221::DEBUG::2012-12-09 13:16:32,018::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3221::DEBUG::2012-12-09 13:16:32,019::task::588::TaskManager.Task::(_updateState) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::moving from state init -> state preparing
>Thread-3221::INFO::2012-12-09 13:16:32,019::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3221::DEBUG::2012-12-09 13:16:32,020::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51d1da9d-cbc9-4e12-96bf-c1375864bd8c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3221::DEBUG::2012-12-09 13:16:32,020::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3221::DEBUG::2012-12-09 13:16:32,020::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3221::DEBUG::2012-12-09 13:16:32,020::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`51d1da9d-cbc9-4e12-96bf-c1375864bd8c`::Granted request
>Thread-3221::DEBUG::2012-12-09 13:16:32,021::task::817::TaskManager.Task::(resourceAcquired) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3221::DEBUG::2012-12-09 13:16:32,021::task::978::TaskManager.Task::(_decref) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::ref 1 aborting False
>Thread-3221::DEBUG::2012-12-09 13:16:32,022::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3221::DEBUG::2012-12-09 13:16:32,022::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:32,094::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:32,135::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162569 s, 63.0 MB/s\n'; <rc> = 0
>Thread-3221::DEBUG::2012-12-09 13:16:32,140::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3221::WARNING::2012-12-09 13:16:32,144::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3221::DEBUG::2012-12-09 13:16:32,145::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3221::WARNING::2012-12-09 13:16:32,146::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3221::INFO::2012-12-09 13:16:32,146::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3221::DEBUG::2012-12-09 13:16:32,146::task::1172::TaskManager.Task::(prepare) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3221::DEBUG::2012-12-09 13:16:32,147::task::588::TaskManager.Task::(_updateState) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::moving from state preparing -> state finished
>Thread-3221::DEBUG::2012-12-09 13:16:32,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3221::DEBUG::2012-12-09 13:16:32,147::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3221::DEBUG::2012-12-09 13:16:32,148::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3221::DEBUG::2012-12-09 13:16:32,148::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3221::DEBUG::2012-12-09 13:16:32,148::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3221::DEBUG::2012-12-09 13:16:32,149::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3221::DEBUG::2012-12-09 13:16:32,149::task::978::TaskManager.Task::(_decref) Task=`1f9d9db0-f9ba-4a5d-b99f-29258fec87f1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:34,146::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:34,186::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166801 s, 61.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:16:34,507::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:34,518::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000375751 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:36,193::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:36,232::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160021 s, 64.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:37,633::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:37,634::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:37,746::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:37,747::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:37,747::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:37,748::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:38,239::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:38,278::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015667 s, 65.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:40,285::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:40,325::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161142 s, 63.5 MB/s\n'; <rc> = 0
>Thread-3226::DEBUG::2012-12-09 13:16:42,013::task::588::TaskManager.Task::(_updateState) Task=`bfd743d6-62c3-4f13-ab95-9646cfdaf17d`::moving from state init -> state preparing
>Thread-3226::INFO::2012-12-09 13:16:42,014::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3226::INFO::2012-12-09 13:16:42,014::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120360851288', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-3226::DEBUG::2012-12-09 13:16:42,014::task::1172::TaskManager.Task::(prepare) Task=`bfd743d6-62c3-4f13-ab95-9646cfdaf17d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120360851288', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-3226::DEBUG::2012-12-09 13:16:42,015::task::588::TaskManager.Task::(_updateState) Task=`bfd743d6-62c3-4f13-ab95-9646cfdaf17d`::moving from state preparing -> state finished
>Thread-3226::DEBUG::2012-12-09 13:16:42,015::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3226::DEBUG::2012-12-09 13:16:42,015::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3226::DEBUG::2012-12-09 13:16:42,016::task::978::TaskManager.Task::(_decref) Task=`bfd743d6-62c3-4f13-ab95-9646cfdaf17d`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:42,332::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:42,372::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168235 s, 60.9 MB/s\n'; <rc> = 0
>Thread-3228::DEBUG::2012-12-09 13:16:42,946::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3228::DEBUG::2012-12-09 13:16:42,947::task::588::TaskManager.Task::(_updateState) Task=`2e034cc6-360a-4f06-a322-ff79d74af2eb`::moving from state init -> state preparing
>Thread-3228::INFO::2012-12-09 13:16:42,947::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3228::INFO::2012-12-09 13:16:42,947::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3228::DEBUG::2012-12-09 13:16:42,947::task::1172::TaskManager.Task::(prepare) Task=`2e034cc6-360a-4f06-a322-ff79d74af2eb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3228::DEBUG::2012-12-09 13:16:42,948::task::588::TaskManager.Task::(_updateState) Task=`2e034cc6-360a-4f06-a322-ff79d74af2eb`::moving from state preparing -> state finished
>Thread-3228::DEBUG::2012-12-09 13:16:42,948::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3228::DEBUG::2012-12-09 13:16:42,948::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3228::DEBUG::2012-12-09 13:16:42,948::task::978::TaskManager.Task::(_decref) Task=`2e034cc6-360a-4f06-a322-ff79d74af2eb`::ref 0 aborting False
>Thread-3229::DEBUG::2012-12-09 13:16:42,960::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3229::DEBUG::2012-12-09 13:16:42,961::task::588::TaskManager.Task::(_updateState) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::moving from state init -> state preparing
>Thread-3229::INFO::2012-12-09 13:16:42,961::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3229::DEBUG::2012-12-09 13:16:42,962::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7e1d9a1-5882-4d71-82bf-ec4a1b7ad41a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3229::DEBUG::2012-12-09 13:16:42,962::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3229::DEBUG::2012-12-09 13:16:42,962::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3229::DEBUG::2012-12-09 13:16:42,962::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7e1d9a1-5882-4d71-82bf-ec4a1b7ad41a`::Granted request
>Thread-3229::DEBUG::2012-12-09 13:16:42,963::task::817::TaskManager.Task::(resourceAcquired) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3229::DEBUG::2012-12-09 13:16:42,963::task::978::TaskManager.Task::(_decref) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::ref 1 aborting False
>Thread-3229::DEBUG::2012-12-09 13:16:42,964::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3229::DEBUG::2012-12-09 13:16:42,964::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3229::DEBUG::2012-12-09 13:16:43,062::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3229::WARNING::2012-12-09 13:16:43,063::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3229::DEBUG::2012-12-09 13:16:43,064::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3229::WARNING::2012-12-09 13:16:43,064::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3229::INFO::2012-12-09 13:16:43,065::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3229::DEBUG::2012-12-09 13:16:43,065::task::1172::TaskManager.Task::(prepare) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3229::DEBUG::2012-12-09 13:16:43,065::task::588::TaskManager.Task::(_updateState) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::moving from state preparing -> state finished
>Thread-3229::DEBUG::2012-12-09 13:16:43,065::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3229::DEBUG::2012-12-09 13:16:43,066::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3229::DEBUG::2012-12-09 13:16:43,066::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3229::DEBUG::2012-12-09 13:16:43,066::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3229::DEBUG::2012-12-09 13:16:43,066::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3229::DEBUG::2012-12-09 13:16:43,067::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3229::DEBUG::2012-12-09 13:16:43,067::task::978::TaskManager.Task::(_decref) Task=`2df4c910-b96b-4ffe-9473-d7b2f47c7650`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:44,379::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:44,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175176 s, 58.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:16:44,521::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:44,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000500927 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:46,428::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:46,471::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185933 s, 55.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:47,749::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:47,749::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:47,848::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:47,850::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:47,850::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:47,850::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:48,478::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:48,518::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167873 s, 61.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:50,525::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:50,565::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166206 s, 61.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:52,573::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:52,612::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165723 s, 61.8 MB/s\n'; <rc> = 0
>Thread-3234::DEBUG::2012-12-09 13:16:52,735::task::588::TaskManager.Task::(_updateState) Task=`37282e64-b0b7-491c-90af-7f1e20eb1820`::moving from state init -> state preparing
>Thread-3234::INFO::2012-12-09 13:16:52,735::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3234::INFO::2012-12-09 13:16:52,736::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00898694992065', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-3234::DEBUG::2012-12-09 13:16:52,736::task::1172::TaskManager.Task::(prepare) Task=`37282e64-b0b7-491c-90af-7f1e20eb1820`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00898694992065', 'lastCheck': '8.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.9', 'code': 358, 'valid': False}}
>Thread-3234::DEBUG::2012-12-09 13:16:52,736::task::588::TaskManager.Task::(_updateState) Task=`37282e64-b0b7-491c-90af-7f1e20eb1820`::moving from state preparing -> state finished
>Thread-3234::DEBUG::2012-12-09 13:16:52,737::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3234::DEBUG::2012-12-09 13:16:52,737::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3234::DEBUG::2012-12-09 13:16:52,737::task::978::TaskManager.Task::(_decref) Task=`37282e64-b0b7-491c-90af-7f1e20eb1820`::ref 0 aborting False
>Thread-3236::DEBUG::2012-12-09 13:16:54,400::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3236::DEBUG::2012-12-09 13:16:54,400::task::588::TaskManager.Task::(_updateState) Task=`fb53dd47-c7f9-442e-b6fb-61339ac8a7ba`::moving from state init -> state preparing
>Thread-3236::INFO::2012-12-09 13:16:54,401::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3236::INFO::2012-12-09 13:16:54,401::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3236::DEBUG::2012-12-09 13:16:54,401::task::1172::TaskManager.Task::(prepare) Task=`fb53dd47-c7f9-442e-b6fb-61339ac8a7ba`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3236::DEBUG::2012-12-09 13:16:54,401::task::588::TaskManager.Task::(_updateState) Task=`fb53dd47-c7f9-442e-b6fb-61339ac8a7ba`::moving from state preparing -> state finished
>Thread-3236::DEBUG::2012-12-09 13:16:54,401::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3236::DEBUG::2012-12-09 13:16:54,402::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3236::DEBUG::2012-12-09 13:16:54,402::task::978::TaskManager.Task::(_decref) Task=`fb53dd47-c7f9-442e-b6fb-61339ac8a7ba`::ref 0 aborting False
>Thread-3237::DEBUG::2012-12-09 13:16:54,424::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3237::DEBUG::2012-12-09 13:16:54,424::task::588::TaskManager.Task::(_updateState) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::moving from state init -> state preparing
>Thread-3237::INFO::2012-12-09 13:16:54,424::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3237::DEBUG::2012-12-09 13:16:54,425::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f73c3a29-4501-4ddb-8c96-3f90f4e0b6f3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3237::DEBUG::2012-12-09 13:16:54,425::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3237::DEBUG::2012-12-09 13:16:54,425::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3237::DEBUG::2012-12-09 13:16:54,426::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`f73c3a29-4501-4ddb-8c96-3f90f4e0b6f3`::Granted request
>Thread-3237::DEBUG::2012-12-09 13:16:54,426::task::817::TaskManager.Task::(resourceAcquired) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3237::DEBUG::2012-12-09 13:16:54,426::task::978::TaskManager.Task::(_decref) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::ref 1 aborting False
>Thread-3237::DEBUG::2012-12-09 13:16:54,427::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3237::DEBUG::2012-12-09 13:16:54,427::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:16:54,533::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-3237::DEBUG::2012-12-09 13:16:54,535::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-50::DEBUG::2012-12-09 13:16:54,546::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00032455 s, 12.6 MB/s\n'; <rc> = 0
>Thread-3237::WARNING::2012-12-09 13:16:54,550::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3237::DEBUG::2012-12-09 13:16:54,551::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3237::WARNING::2012-12-09 13:16:54,553::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3237::INFO::2012-12-09 13:16:54,554::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3237::DEBUG::2012-12-09 13:16:54,554::task::1172::TaskManager.Task::(prepare) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3237::DEBUG::2012-12-09 13:16:54,555::task::588::TaskManager.Task::(_updateState) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::moving from state preparing -> state finished
>Thread-3237::DEBUG::2012-12-09 13:16:54,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3237::DEBUG::2012-12-09 13:16:54,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3237::DEBUG::2012-12-09 13:16:54,556::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3237::DEBUG::2012-12-09 13:16:54,557::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3237::DEBUG::2012-12-09 13:16:54,557::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3237::DEBUG::2012-12-09 13:16:54,557::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3237::DEBUG::2012-12-09 13:16:54,557::task::978::TaskManager.Task::(_decref) Task=`430f177c-7138-4f5b-821c-e9799d25f9b9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:16:54,619::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:54,664::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164489 s, 62.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:16:56,671::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:56,713::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173665 s, 59.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:16:57,851::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:16:57,852::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:16:57,951::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:16:57,952::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:16:57,952::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:16:57,953::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:16:58,720::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:16:58,768::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161254 s, 63.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:00,775::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:00,814::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160432 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:02,820::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:02,869::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177334 s, 57.7 MB/s\n'; <rc> = 0
>Thread-3242::DEBUG::2012-12-09 13:17:03,622::task::588::TaskManager.Task::(_updateState) Task=`8329d690-1725-4073-8894-089f79d1696d`::moving from state init -> state preparing
>Thread-3242::INFO::2012-12-09 13:17:03,623::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3242::INFO::2012-12-09 13:17:03,623::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140528678894', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-3242::DEBUG::2012-12-09 13:17:03,623::task::1172::TaskManager.Task::(prepare) Task=`8329d690-1725-4073-8894-089f79d1696d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140528678894', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.7', 'code': 358, 'valid': False}}
>Thread-3242::DEBUG::2012-12-09 13:17:03,623::task::588::TaskManager.Task::(_updateState) Task=`8329d690-1725-4073-8894-089f79d1696d`::moving from state preparing -> state finished
>Thread-3242::DEBUG::2012-12-09 13:17:03,623::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3242::DEBUG::2012-12-09 13:17:03,624::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3242::DEBUG::2012-12-09 13:17:03,624::task::978::TaskManager.Task::(_decref) Task=`8329d690-1725-4073-8894-089f79d1696d`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:17:04,555::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:04,568::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000520394 s, 7.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:04,877::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:04,947::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304142 s, 33.7 MB/s\n'; <rc> = 0
>Thread-3244::DEBUG::2012-12-09 13:17:05,804::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3244::DEBUG::2012-12-09 13:17:05,805::task::588::TaskManager.Task::(_updateState) Task=`5b13b3a9-8455-4d31-a585-02dc2a37ff0b`::moving from state init -> state preparing
>Thread-3244::INFO::2012-12-09 13:17:05,805::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3244::INFO::2012-12-09 13:17:05,805::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3244::DEBUG::2012-12-09 13:17:05,805::task::1172::TaskManager.Task::(prepare) Task=`5b13b3a9-8455-4d31-a585-02dc2a37ff0b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3244::DEBUG::2012-12-09 13:17:05,806::task::588::TaskManager.Task::(_updateState) Task=`5b13b3a9-8455-4d31-a585-02dc2a37ff0b`::moving from state preparing -> state finished
>Thread-3244::DEBUG::2012-12-09 13:17:05,806::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3244::DEBUG::2012-12-09 13:17:05,806::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3244::DEBUG::2012-12-09 13:17:05,806::task::978::TaskManager.Task::(_decref) Task=`5b13b3a9-8455-4d31-a585-02dc2a37ff0b`::ref 0 aborting False
>Thread-3245::DEBUG::2012-12-09 13:17:05,822::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3245::DEBUG::2012-12-09 13:17:05,822::task::588::TaskManager.Task::(_updateState) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::moving from state init -> state preparing
>Thread-3245::INFO::2012-12-09 13:17:05,822::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3245::DEBUG::2012-12-09 13:17:05,823::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0d09dc02-5a97-4051-bfa1-8b32da38626d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3245::DEBUG::2012-12-09 13:17:05,823::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3245::DEBUG::2012-12-09 13:17:05,823::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3245::DEBUG::2012-12-09 13:17:05,823::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0d09dc02-5a97-4051-bfa1-8b32da38626d`::Granted request
>Thread-3245::DEBUG::2012-12-09 13:17:05,824::task::817::TaskManager.Task::(resourceAcquired) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3245::DEBUG::2012-12-09 13:17:05,824::task::978::TaskManager.Task::(_decref) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::ref 1 aborting False
>Thread-3245::DEBUG::2012-12-09 13:17:05,825::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3245::DEBUG::2012-12-09 13:17:05,825::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3245::DEBUG::2012-12-09 13:17:05,925::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3245::WARNING::2012-12-09 13:17:05,926::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3245::DEBUG::2012-12-09 13:17:05,926::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3245::WARNING::2012-12-09 13:17:05,927::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3245::INFO::2012-12-09 13:17:05,927::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3245::DEBUG::2012-12-09 13:17:05,928::task::1172::TaskManager.Task::(prepare) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3245::DEBUG::2012-12-09 13:17:05,928::task::588::TaskManager.Task::(_updateState) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::moving from state preparing -> state finished
>Thread-3245::DEBUG::2012-12-09 13:17:05,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3245::DEBUG::2012-12-09 13:17:05,928::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3245::DEBUG::2012-12-09 13:17:05,928::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3245::DEBUG::2012-12-09 13:17:05,929::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3245::DEBUG::2012-12-09 13:17:05,929::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3245::DEBUG::2012-12-09 13:17:05,929::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3245::DEBUG::2012-12-09 13:17:05,929::task::978::TaskManager.Task::(_decref) Task=`ddb7163e-3d8f-4634-892d-d138890692d7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:06,956::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:07,023::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263921 s, 38.8 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:17:07,953::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:07,954::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:08,060::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:08,061::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:08,062::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:08,062::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:17:09,032::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:09,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020179 s, 50.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:11,084::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:11,125::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181293 s, 56.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:13,132::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:13,171::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155964 s, 65.7 MB/s\n'; <rc> = 0
>Thread-3250::DEBUG::2012-12-09 13:17:14,238::task::588::TaskManager.Task::(_updateState) Task=`8dc8b05a-b6bb-4afc-b611-2fe368c490f9`::moving from state init -> state preparing
>Thread-3250::INFO::2012-12-09 13:17:14,238::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3250::INFO::2012-12-09 13:17:14,238::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136640071869', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-3250::DEBUG::2012-12-09 13:17:14,239::task::1172::TaskManager.Task::(prepare) Task=`8dc8b05a-b6bb-4afc-b611-2fe368c490f9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0136640071869', 'lastCheck': '9.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.2', 'code': 358, 'valid': False}}
>Thread-3250::DEBUG::2012-12-09 13:17:14,239::task::588::TaskManager.Task::(_updateState) Task=`8dc8b05a-b6bb-4afc-b611-2fe368c490f9`::moving from state preparing -> state finished
>Thread-3250::DEBUG::2012-12-09 13:17:14,239::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3250::DEBUG::2012-12-09 13:17:14,239::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3250::DEBUG::2012-12-09 13:17:14,239::task::978::TaskManager.Task::(_decref) Task=`8dc8b05a-b6bb-4afc-b611-2fe368c490f9`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:17:14,573::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:14,584::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000431443 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:15,177::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:15,218::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170335 s, 60.1 MB/s\n'; <rc> = 0
>Thread-3252::DEBUG::2012-12-09 13:17:16,568::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3252::DEBUG::2012-12-09 13:17:16,568::task::588::TaskManager.Task::(_updateState) Task=`65951ed1-d082-4175-b2e2-b844c116a221`::moving from state init -> state preparing
>Thread-3252::INFO::2012-12-09 13:17:16,569::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3252::INFO::2012-12-09 13:17:16,569::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3252::DEBUG::2012-12-09 13:17:16,569::task::1172::TaskManager.Task::(prepare) Task=`65951ed1-d082-4175-b2e2-b844c116a221`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3252::DEBUG::2012-12-09 13:17:16,569::task::588::TaskManager.Task::(_updateState) Task=`65951ed1-d082-4175-b2e2-b844c116a221`::moving from state preparing -> state finished
>Thread-3252::DEBUG::2012-12-09 13:17:16,570::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3252::DEBUG::2012-12-09 13:17:16,570::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3252::DEBUG::2012-12-09 13:17:16,570::task::978::TaskManager.Task::(_decref) Task=`65951ed1-d082-4175-b2e2-b844c116a221`::ref 0 aborting False
>Thread-3253::DEBUG::2012-12-09 13:17:16,585::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3253::DEBUG::2012-12-09 13:17:16,585::task::588::TaskManager.Task::(_updateState) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::moving from state init -> state preparing
>Thread-3253::INFO::2012-12-09 13:17:16,585::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3253::DEBUG::2012-12-09 13:17:16,586::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a322cd4-940a-4b82-9a37-817ea6bf8e68`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3253::DEBUG::2012-12-09 13:17:16,586::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3253::DEBUG::2012-12-09 13:17:16,586::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3253::DEBUG::2012-12-09 13:17:16,586::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`0a322cd4-940a-4b82-9a37-817ea6bf8e68`::Granted request
>Thread-3253::DEBUG::2012-12-09 13:17:16,587::task::817::TaskManager.Task::(resourceAcquired) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3253::DEBUG::2012-12-09 13:17:16,587::task::978::TaskManager.Task::(_decref) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::ref 1 aborting False
>Thread-3253::DEBUG::2012-12-09 13:17:16,588::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3253::DEBUG::2012-12-09 13:17:16,588::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3253::DEBUG::2012-12-09 13:17:16,686::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3253::WARNING::2012-12-09 13:17:16,687::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3253::DEBUG::2012-12-09 13:17:16,687::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3253::WARNING::2012-12-09 13:17:16,688::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3253::INFO::2012-12-09 13:17:16,688::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3253::DEBUG::2012-12-09 13:17:16,688::task::1172::TaskManager.Task::(prepare) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3253::DEBUG::2012-12-09 13:17:16,689::task::588::TaskManager.Task::(_updateState) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::moving from state preparing -> state finished
>Thread-3253::DEBUG::2012-12-09 13:17:16,689::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3253::DEBUG::2012-12-09 13:17:16,689::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3253::DEBUG::2012-12-09 13:17:16,689::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3253::DEBUG::2012-12-09 13:17:16,690::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3253::DEBUG::2012-12-09 13:17:16,690::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3253::DEBUG::2012-12-09 13:17:16,690::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3253::DEBUG::2012-12-09 13:17:16,690::task::978::TaskManager.Task::(_decref) Task=`488c02e4-2d62-4c0a-90c3-b547e0bbf371`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:17,224::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:17,263::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157669 s, 64.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:17:18,063::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:18,063::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:18,161::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:18,163::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:18,163::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:18,163::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:17:19,270::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:19,311::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160274 s, 63.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:21,318::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:21,364::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016493 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:23,371::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:23,421::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235178 s, 43.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:17:24,587::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:24,599::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000373019 s, 11.0 MB/s\n'; <rc> = 0
>Thread-3258::DEBUG::2012-12-09 13:17:25,427::task::588::TaskManager.Task::(_updateState) Task=`dbbe3903-e068-4935-a937-04981c41c866`::moving from state init -> state preparing
>Thread-3258::INFO::2012-12-09 13:17:25,427::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3258::INFO::2012-12-09 13:17:25,428::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120098590851', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Dummy-321::DEBUG::2012-12-09 13:17:25,428::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3258::DEBUG::2012-12-09 13:17:25,428::task::1172::TaskManager.Task::(prepare) Task=`dbbe3903-e068-4935-a937-04981c41c866`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120098590851', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.3', 'code': 358, 'valid': False}}
>Thread-3258::DEBUG::2012-12-09 13:17:25,434::task::588::TaskManager.Task::(_updateState) Task=`dbbe3903-e068-4935-a937-04981c41c866`::moving from state preparing -> state finished
>Thread-3258::DEBUG::2012-12-09 13:17:25,436::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3258::DEBUG::2012-12-09 13:17:25,437::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3258::DEBUG::2012-12-09 13:17:25,438::task::978::TaskManager.Task::(_decref) Task=`dbbe3903-e068-4935-a937-04981c41c866`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:25,485::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0327124 s, 31.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:27,492::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:27,549::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234999 s, 43.6 MB/s\n'; <rc> = 0
>Thread-3260::DEBUG::2012-12-09 13:17:27,749::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3260::DEBUG::2012-12-09 13:17:27,749::task::588::TaskManager.Task::(_updateState) Task=`be4a7abb-5ea2-459a-8cfa-52219b00097d`::moving from state init -> state preparing
>Thread-3260::INFO::2012-12-09 13:17:27,749::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3260::INFO::2012-12-09 13:17:27,750::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3260::DEBUG::2012-12-09 13:17:27,750::task::1172::TaskManager.Task::(prepare) Task=`be4a7abb-5ea2-459a-8cfa-52219b00097d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3260::DEBUG::2012-12-09 13:17:27,750::task::588::TaskManager.Task::(_updateState) Task=`be4a7abb-5ea2-459a-8cfa-52219b00097d`::moving from state preparing -> state finished
>Thread-3260::DEBUG::2012-12-09 13:17:27,750::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3260::DEBUG::2012-12-09 13:17:27,750::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3260::DEBUG::2012-12-09 13:17:27,751::task::978::TaskManager.Task::(_decref) Task=`be4a7abb-5ea2-459a-8cfa-52219b00097d`::ref 0 aborting False
>Thread-3261::DEBUG::2012-12-09 13:17:27,763::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3261::DEBUG::2012-12-09 13:17:27,764::task::588::TaskManager.Task::(_updateState) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::moving from state init -> state preparing
>Thread-3261::INFO::2012-12-09 13:17:27,764::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3261::DEBUG::2012-12-09 13:17:27,764::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c920abe0-9ebd-4ff6-a53a-53a616288ce4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3261::DEBUG::2012-12-09 13:17:27,765::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3261::DEBUG::2012-12-09 13:17:27,765::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3261::DEBUG::2012-12-09 13:17:27,765::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`c920abe0-9ebd-4ff6-a53a-53a616288ce4`::Granted request
>Thread-3261::DEBUG::2012-12-09 13:17:27,765::task::817::TaskManager.Task::(resourceAcquired) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3261::DEBUG::2012-12-09 13:17:27,766::task::978::TaskManager.Task::(_decref) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::ref 1 aborting False
>Thread-3261::DEBUG::2012-12-09 13:17:27,766::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3261::DEBUG::2012-12-09 13:17:27,767::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3261::DEBUG::2012-12-09 13:17:27,874::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3261::WARNING::2012-12-09 13:17:27,875::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3261::DEBUG::2012-12-09 13:17:27,875::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3261::WARNING::2012-12-09 13:17:27,876::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3261::INFO::2012-12-09 13:17:27,876::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3261::DEBUG::2012-12-09 13:17:27,876::task::1172::TaskManager.Task::(prepare) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3261::DEBUG::2012-12-09 13:17:27,877::task::588::TaskManager.Task::(_updateState) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::moving from state preparing -> state finished
>Thread-3261::DEBUG::2012-12-09 13:17:27,877::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3261::DEBUG::2012-12-09 13:17:27,877::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3261::DEBUG::2012-12-09 13:17:27,877::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3261::DEBUG::2012-12-09 13:17:27,877::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3261::DEBUG::2012-12-09 13:17:27,878::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3261::DEBUG::2012-12-09 13:17:27,878::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3261::DEBUG::2012-12-09 13:17:27,878::task::978::TaskManager.Task::(_decref) Task=`46109c24-3147-48db-a8ac-83d755e95e90`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:17:28,164::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:28,165::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:28,264::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:28,265::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:28,266::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:28,266::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:17:29,556::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:29,596::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160599 s, 63.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:31,603::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:31,644::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163791 s, 62.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:33,651::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:33,694::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184423 s, 55.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:17:34,602::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:34,615::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000440982 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:35,701::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:35,742::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171513 s, 59.7 MB/s\n'; <rc> = 0
>Thread-3266::DEBUG::2012-12-09 13:17:36,150::task::588::TaskManager.Task::(_updateState) Task=`f063f26a-1a97-4042-bbeb-88f9b2e3f1e1`::moving from state init -> state preparing
>Thread-3266::INFO::2012-12-09 13:17:36,150::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3266::INFO::2012-12-09 13:17:36,150::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138750076294', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-3266::DEBUG::2012-12-09 13:17:36,150::task::1172::TaskManager.Task::(prepare) Task=`f063f26a-1a97-4042-bbeb-88f9b2e3f1e1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138750076294', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.9', 'code': 358, 'valid': False}}
>Thread-3266::DEBUG::2012-12-09 13:17:36,151::task::588::TaskManager.Task::(_updateState) Task=`f063f26a-1a97-4042-bbeb-88f9b2e3f1e1`::moving from state preparing -> state finished
>Thread-3266::DEBUG::2012-12-09 13:17:36,151::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3266::DEBUG::2012-12-09 13:17:36,151::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3266::DEBUG::2012-12-09 13:17:36,151::task::978::TaskManager.Task::(_decref) Task=`f063f26a-1a97-4042-bbeb-88f9b2e3f1e1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:37,749::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:37,806::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212642 s, 48.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:17:38,267::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:38,268::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:38,374::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:38,376::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:38,376::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:38,376::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:17:39,817::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:39,858::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180084 s, 56.9 MB/s\n'; <rc> = 0
>Thread-3269::DEBUG::2012-12-09 13:17:40,325::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3269::DEBUG::2012-12-09 13:17:40,326::task::588::TaskManager.Task::(_updateState) Task=`e2974320-e570-49af-8fd4-7f8d29040dc5`::moving from state init -> state preparing
>Thread-3269::INFO::2012-12-09 13:17:40,326::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3269::INFO::2012-12-09 13:17:40,326::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3269::DEBUG::2012-12-09 13:17:40,327::task::1172::TaskManager.Task::(prepare) Task=`e2974320-e570-49af-8fd4-7f8d29040dc5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3269::DEBUG::2012-12-09 13:17:40,327::task::588::TaskManager.Task::(_updateState) Task=`e2974320-e570-49af-8fd4-7f8d29040dc5`::moving from state preparing -> state finished
>Thread-3269::DEBUG::2012-12-09 13:17:40,327::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3269::DEBUG::2012-12-09 13:17:40,327::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3269::DEBUG::2012-12-09 13:17:40,328::task::978::TaskManager.Task::(_decref) Task=`e2974320-e570-49af-8fd4-7f8d29040dc5`::ref 0 aborting False
>Thread-3270::DEBUG::2012-12-09 13:17:40,343::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3270::DEBUG::2012-12-09 13:17:40,344::task::588::TaskManager.Task::(_updateState) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::moving from state init -> state preparing
>Thread-3270::INFO::2012-12-09 13:17:40,344::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3270::DEBUG::2012-12-09 13:17:40,344::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d66192e6-a0b0-42ec-9788-7895eb6bf28b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3270::DEBUG::2012-12-09 13:17:40,344::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3270::DEBUG::2012-12-09 13:17:40,345::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3270::DEBUG::2012-12-09 13:17:40,345::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d66192e6-a0b0-42ec-9788-7895eb6bf28b`::Granted request
>Thread-3270::DEBUG::2012-12-09 13:17:40,345::task::817::TaskManager.Task::(resourceAcquired) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3270::DEBUG::2012-12-09 13:17:40,345::task::978::TaskManager.Task::(_decref) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::ref 1 aborting False
>Thread-3270::DEBUG::2012-12-09 13:17:40,346::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3270::DEBUG::2012-12-09 13:17:40,347::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3270::DEBUG::2012-12-09 13:17:40,455::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3270::WARNING::2012-12-09 13:17:40,456::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3270::DEBUG::2012-12-09 13:17:40,457::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3270::WARNING::2012-12-09 13:17:40,457::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3270::INFO::2012-12-09 13:17:40,458::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3270::DEBUG::2012-12-09 13:17:40,458::task::1172::TaskManager.Task::(prepare) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3270::DEBUG::2012-12-09 13:17:40,458::task::588::TaskManager.Task::(_updateState) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::moving from state preparing -> state finished
>Thread-3270::DEBUG::2012-12-09 13:17:40,458::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3270::DEBUG::2012-12-09 13:17:40,459::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3270::DEBUG::2012-12-09 13:17:40,459::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3270::DEBUG::2012-12-09 13:17:40,459::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3270::DEBUG::2012-12-09 13:17:40,459::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3270::DEBUG::2012-12-09 13:17:40,459::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3270::DEBUG::2012-12-09 13:17:40,460::task::978::TaskManager.Task::(_decref) Task=`39d953da-10d1-46df-ad9d-b4d642614953`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:41,865::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:41,907::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166061 s, 61.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:43,914::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:43,958::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194931 s, 52.5 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:17:44,619::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:44,628::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000365174 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:45,965::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:46,012::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184662 s, 55.5 MB/s\n'; <rc> = 0
>Thread-3274::DEBUG::2012-12-09 13:17:47,310::task::588::TaskManager.Task::(_updateState) Task=`7e25e68a-c5fa-4f6f-9623-1b55c18450d1`::moving from state init -> state preparing
>Thread-3274::INFO::2012-12-09 13:17:47,311::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3274::INFO::2012-12-09 13:17:47,311::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00908994674683', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-3274::DEBUG::2012-12-09 13:17:47,311::task::1172::TaskManager.Task::(prepare) Task=`7e25e68a-c5fa-4f6f-9623-1b55c18450d1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00908994674683', 'lastCheck': '2.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.9', 'code': 358, 'valid': False}}
>Thread-3274::DEBUG::2012-12-09 13:17:47,311::task::588::TaskManager.Task::(_updateState) Task=`7e25e68a-c5fa-4f6f-9623-1b55c18450d1`::moving from state preparing -> state finished
>Thread-3274::DEBUG::2012-12-09 13:17:47,312::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3274::DEBUG::2012-12-09 13:17:47,312::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3274::DEBUG::2012-12-09 13:17:47,312::task::978::TaskManager.Task::(_decref) Task=`7e25e68a-c5fa-4f6f-9623-1b55c18450d1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:48,020::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:48,061::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016436 s, 62.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:17:48,377::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:48,378::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:48,476::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:48,477::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:48,477::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:48,478::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:17:50,068::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:50,108::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167806 s, 61.0 MB/s\n'; <rc> = 0
>Thread-3277::DEBUG::2012-12-09 13:17:51,708::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3277::DEBUG::2012-12-09 13:17:51,708::task::588::TaskManager.Task::(_updateState) Task=`3463d4c0-e2c4-44e5-984b-83bfc60402e0`::moving from state init -> state preparing
>Thread-3277::INFO::2012-12-09 13:17:51,709::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3277::INFO::2012-12-09 13:17:51,709::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3277::DEBUG::2012-12-09 13:17:51,709::task::1172::TaskManager.Task::(prepare) Task=`3463d4c0-e2c4-44e5-984b-83bfc60402e0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3277::DEBUG::2012-12-09 13:17:51,709::task::588::TaskManager.Task::(_updateState) Task=`3463d4c0-e2c4-44e5-984b-83bfc60402e0`::moving from state preparing -> state finished
>Thread-3277::DEBUG::2012-12-09 13:17:51,709::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3277::DEBUG::2012-12-09 13:17:51,710::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3277::DEBUG::2012-12-09 13:17:51,710::task::978::TaskManager.Task::(_decref) Task=`3463d4c0-e2c4-44e5-984b-83bfc60402e0`::ref 0 aborting False
>Thread-3278::DEBUG::2012-12-09 13:17:51,722::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3278::DEBUG::2012-12-09 13:17:51,723::task::588::TaskManager.Task::(_updateState) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::moving from state init -> state preparing
>Thread-3278::INFO::2012-12-09 13:17:51,723::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3278::DEBUG::2012-12-09 13:17:51,723::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6c394059-f179-4bf6-a563-6baf2e722cae`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3278::DEBUG::2012-12-09 13:17:51,723::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3278::DEBUG::2012-12-09 13:17:51,724::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3278::DEBUG::2012-12-09 13:17:51,724::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6c394059-f179-4bf6-a563-6baf2e722cae`::Granted request
>Thread-3278::DEBUG::2012-12-09 13:17:51,724::task::817::TaskManager.Task::(resourceAcquired) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3278::DEBUG::2012-12-09 13:17:51,724::task::978::TaskManager.Task::(_decref) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::ref 1 aborting False
>Thread-3278::DEBUG::2012-12-09 13:17:51,725::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3278::DEBUG::2012-12-09 13:17:51,725::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3278::DEBUG::2012-12-09 13:17:51,824::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3278::WARNING::2012-12-09 13:17:51,825::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3278::DEBUG::2012-12-09 13:17:51,825::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3278::WARNING::2012-12-09 13:17:51,826::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3278::INFO::2012-12-09 13:17:51,826::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3278::DEBUG::2012-12-09 13:17:51,826::task::1172::TaskManager.Task::(prepare) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3278::DEBUG::2012-12-09 13:17:51,826::task::588::TaskManager.Task::(_updateState) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::moving from state preparing -> state finished
>Thread-3278::DEBUG::2012-12-09 13:17:51,827::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3278::DEBUG::2012-12-09 13:17:51,827::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3278::DEBUG::2012-12-09 13:17:51,827::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3278::DEBUG::2012-12-09 13:17:51,827::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3278::DEBUG::2012-12-09 13:17:51,828::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3278::DEBUG::2012-12-09 13:17:51,828::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3278::DEBUG::2012-12-09 13:17:51,828::task::978::TaskManager.Task::(_decref) Task=`71c96477-0c14-42c1-9547-20a4ecd958c2`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:52,114::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:52,154::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163296 s, 62.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:54,165::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:54,217::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176249 s, 58.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:17:54,631::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:17:54,643::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000451377 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:17:56,226::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:56,267::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165122 s, 62.0 MB/s\n'; <rc> = 0
>Thread-3282::DEBUG::2012-12-09 13:17:58,091::task::588::TaskManager.Task::(_updateState) Task=`9d09cedd-f963-4251-807b-b8f4d1dff1c7`::moving from state init -> state preparing
>Thread-3282::INFO::2012-12-09 13:17:58,091::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3282::INFO::2012-12-09 13:17:58,092::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128149986267', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-3282::DEBUG::2012-12-09 13:17:58,092::task::1172::TaskManager.Task::(prepare) Task=`9d09cedd-f963-4251-807b-b8f4d1dff1c7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0128149986267', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.6', 'code': 358, 'valid': False}}
>Thread-3282::DEBUG::2012-12-09 13:17:58,092::task::588::TaskManager.Task::(_updateState) Task=`9d09cedd-f963-4251-807b-b8f4d1dff1c7`::moving from state preparing -> state finished
>Thread-3282::DEBUG::2012-12-09 13:17:58,092::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3282::DEBUG::2012-12-09 13:17:58,092::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3282::DEBUG::2012-12-09 13:17:58,093::task::978::TaskManager.Task::(_decref) Task=`9d09cedd-f963-4251-807b-b8f4d1dff1c7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:17:58,274::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:17:58,314::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172443 s, 59.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:17:58,479::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:17:58,479::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:17:58,572::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:17:58,574::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:17:58,574::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:17:58,574::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:18:00,319::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:00,360::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172359 s, 59.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:02,369::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:02,410::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177849 s, 57.6 MB/s\n'; <rc> = 0
>Thread-3286::DEBUG::2012-12-09 13:18:03,051::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3286::DEBUG::2012-12-09 13:18:03,051::task::588::TaskManager.Task::(_updateState) Task=`6b062cd1-3a99-49c6-ae05-1431c34fa9fd`::moving from state init -> state preparing
>Thread-3286::INFO::2012-12-09 13:18:03,052::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3286::INFO::2012-12-09 13:18:03,052::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3286::DEBUG::2012-12-09 13:18:03,052::task::1172::TaskManager.Task::(prepare) Task=`6b062cd1-3a99-49c6-ae05-1431c34fa9fd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3286::DEBUG::2012-12-09 13:18:03,052::task::588::TaskManager.Task::(_updateState) Task=`6b062cd1-3a99-49c6-ae05-1431c34fa9fd`::moving from state preparing -> state finished
>Thread-3286::DEBUG::2012-12-09 13:18:03,053::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3286::DEBUG::2012-12-09 13:18:03,053::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3286::DEBUG::2012-12-09 13:18:03,053::task::978::TaskManager.Task::(_decref) Task=`6b062cd1-3a99-49c6-ae05-1431c34fa9fd`::ref 0 aborting False
>Thread-3287::DEBUG::2012-12-09 13:18:03,066::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3287::DEBUG::2012-12-09 13:18:03,067::task::588::TaskManager.Task::(_updateState) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::moving from state init -> state preparing
>Thread-3287::INFO::2012-12-09 13:18:03,067::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3287::DEBUG::2012-12-09 13:18:03,067::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b45f632d-9461-44c8-8136-ff238ed198db`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3287::DEBUG::2012-12-09 13:18:03,068::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3287::DEBUG::2012-12-09 13:18:03,068::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3287::DEBUG::2012-12-09 13:18:03,068::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b45f632d-9461-44c8-8136-ff238ed198db`::Granted request
>Thread-3287::DEBUG::2012-12-09 13:18:03,069::task::817::TaskManager.Task::(resourceAcquired) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3287::DEBUG::2012-12-09 13:18:03,069::task::978::TaskManager.Task::(_decref) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::ref 1 aborting False
>Thread-3287::DEBUG::2012-12-09 13:18:03,070::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3287::DEBUG::2012-12-09 13:18:03,070::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3287::DEBUG::2012-12-09 13:18:03,174::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3287::WARNING::2012-12-09 13:18:03,175::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3287::DEBUG::2012-12-09 13:18:03,175::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3287::WARNING::2012-12-09 13:18:03,176::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3287::INFO::2012-12-09 13:18:03,176::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3287::DEBUG::2012-12-09 13:18:03,176::task::1172::TaskManager.Task::(prepare) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3287::DEBUG::2012-12-09 13:18:03,176::task::588::TaskManager.Task::(_updateState) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::moving from state preparing -> state finished
>Thread-3287::DEBUG::2012-12-09 13:18:03,177::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3287::DEBUG::2012-12-09 13:18:03,177::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3287::DEBUG::2012-12-09 13:18:03,177::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3287::DEBUG::2012-12-09 13:18:03,177::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3287::DEBUG::2012-12-09 13:18:03,177::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3287::DEBUG::2012-12-09 13:18:03,178::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3287::DEBUG::2012-12-09 13:18:03,178::task::978::TaskManager.Task::(_decref) Task=`2851bf9a-4eaa-48a2-a201-3284cf0f343a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:04,417::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:04,460::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193562 s, 52.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:18:04,646::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:04,656::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000420416 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:06,467::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:06,507::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016541 s, 61.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:08,515::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:08,555::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169348 s, 60.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:18:08,575::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:08,576::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:08,672::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:08,673::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:08,674::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:08,674::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3290::DEBUG::2012-12-09 13:18:08,802::task::588::TaskManager.Task::(_updateState) Task=`21e7927c-0e35-4f24-b376-947918433269`::moving from state init -> state preparing
>Thread-3290::INFO::2012-12-09 13:18:08,803::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3290::INFO::2012-12-09 13:18:08,803::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102269649506', 'lastCheck': '4.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-3290::DEBUG::2012-12-09 13:18:08,803::task::1172::TaskManager.Task::(prepare) Task=`21e7927c-0e35-4f24-b376-947918433269`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102269649506', 'lastCheck': '4.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.1', 'code': 358, 'valid': False}}
>Thread-3290::DEBUG::2012-12-09 13:18:08,803::task::588::TaskManager.Task::(_updateState) Task=`21e7927c-0e35-4f24-b376-947918433269`::moving from state preparing -> state finished
>Thread-3290::DEBUG::2012-12-09 13:18:08,804::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3290::DEBUG::2012-12-09 13:18:08,804::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3290::DEBUG::2012-12-09 13:18:08,804::task::978::TaskManager.Task::(_decref) Task=`21e7927c-0e35-4f24-b376-947918433269`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:10,563::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:10,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024298 s, 42.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:12,618::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:12,658::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163368 s, 62.7 MB/s\n'; <rc> = 0
>Thread-3294::DEBUG::2012-12-09 13:18:14,012::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3294::DEBUG::2012-12-09 13:18:14,013::task::588::TaskManager.Task::(_updateState) Task=`fc249d0e-5221-4b5e-b7df-dd23d36f4172`::moving from state init -> state preparing
>Thread-3294::INFO::2012-12-09 13:18:14,013::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3294::INFO::2012-12-09 13:18:14,014::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3294::DEBUG::2012-12-09 13:18:14,014::task::1172::TaskManager.Task::(prepare) Task=`fc249d0e-5221-4b5e-b7df-dd23d36f4172`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3294::DEBUG::2012-12-09 13:18:14,014::task::588::TaskManager.Task::(_updateState) Task=`fc249d0e-5221-4b5e-b7df-dd23d36f4172`::moving from state preparing -> state finished
>Thread-3294::DEBUG::2012-12-09 13:18:14,014::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3294::DEBUG::2012-12-09 13:18:14,014::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3294::DEBUG::2012-12-09 13:18:14,015::task::978::TaskManager.Task::(_decref) Task=`fc249d0e-5221-4b5e-b7df-dd23d36f4172`::ref 0 aborting False
>Thread-3295::DEBUG::2012-12-09 13:18:14,030::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3295::DEBUG::2012-12-09 13:18:14,031::task::588::TaskManager.Task::(_updateState) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::moving from state init -> state preparing
>Thread-3295::INFO::2012-12-09 13:18:14,031::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3295::DEBUG::2012-12-09 13:18:14,032::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3c2e9c82-388f-47ae-a616-17c800464ee5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3295::DEBUG::2012-12-09 13:18:14,032::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3295::DEBUG::2012-12-09 13:18:14,032::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3295::DEBUG::2012-12-09 13:18:14,032::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3c2e9c82-388f-47ae-a616-17c800464ee5`::Granted request
>Thread-3295::DEBUG::2012-12-09 13:18:14,033::task::817::TaskManager.Task::(resourceAcquired) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3295::DEBUG::2012-12-09 13:18:14,033::task::978::TaskManager.Task::(_decref) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::ref 1 aborting False
>Thread-3295::DEBUG::2012-12-09 13:18:14,034::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3295::DEBUG::2012-12-09 13:18:14,034::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3295::DEBUG::2012-12-09 13:18:14,131::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3295::WARNING::2012-12-09 13:18:14,132::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3295::DEBUG::2012-12-09 13:18:14,133::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3295::WARNING::2012-12-09 13:18:14,133::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3295::INFO::2012-12-09 13:18:14,134::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3295::DEBUG::2012-12-09 13:18:14,134::task::1172::TaskManager.Task::(prepare) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3295::DEBUG::2012-12-09 13:18:14,134::task::588::TaskManager.Task::(_updateState) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::moving from state preparing -> state finished
>Thread-3295::DEBUG::2012-12-09 13:18:14,134::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3295::DEBUG::2012-12-09 13:18:14,134::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3295::DEBUG::2012-12-09 13:18:14,135::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3295::DEBUG::2012-12-09 13:18:14,135::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3295::DEBUG::2012-12-09 13:18:14,135::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3295::DEBUG::2012-12-09 13:18:14,135::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3295::DEBUG::2012-12-09 13:18:14,136::task::978::TaskManager.Task::(_decref) Task=`bf10622b-8b43-4426-87a5-1d285be580ad`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:18:14,659::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:14,664::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:14,672::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00040817 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:14,709::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0217621 s, 47.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:16,716::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:16,757::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171007 s, 59.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:18:18,675::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:18,676::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:18,764::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:18,775::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:18,776::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:18,777::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:18,777::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:18:18,816::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184649 s, 55.5 MB/s\n'; <rc> = 0
>Thread-3298::DEBUG::2012-12-09 13:18:19,680::task::588::TaskManager.Task::(_updateState) Task=`26f58eca-027c-477c-9e01-b3bbf7c8feb0`::moving from state init -> state preparing
>Thread-3298::INFO::2012-12-09 13:18:19,680::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3298::INFO::2012-12-09 13:18:19,681::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140171051025', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-3298::DEBUG::2012-12-09 13:18:19,681::task::1172::TaskManager.Task::(prepare) Task=`26f58eca-027c-477c-9e01-b3bbf7c8feb0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0140171051025', 'lastCheck': '5.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '0.9', 'code': 358, 'valid': False}}
>Thread-3298::DEBUG::2012-12-09 13:18:19,681::task::588::TaskManager.Task::(_updateState) Task=`26f58eca-027c-477c-9e01-b3bbf7c8feb0`::moving from state preparing -> state finished
>Thread-3298::DEBUG::2012-12-09 13:18:19,681::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3298::DEBUG::2012-12-09 13:18:19,681::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3298::DEBUG::2012-12-09 13:18:19,682::task::978::TaskManager.Task::(_decref) Task=`26f58eca-027c-477c-9e01-b3bbf7c8feb0`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:20,823::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:20,864::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165796 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:22,871::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:22,915::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201497 s, 50.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:18:24,679::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:24,691::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000548928 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:24,920::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3302::DEBUG::2012-12-09 13:18:24,926::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3302::DEBUG::2012-12-09 13:18:24,927::task::588::TaskManager.Task::(_updateState) Task=`48f264e1-89a8-4986-a602-c160d48d4bd8`::moving from state init -> state preparing
>Thread-3302::INFO::2012-12-09 13:18:24,927::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3302::INFO::2012-12-09 13:18:24,928::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3302::DEBUG::2012-12-09 13:18:24,928::task::1172::TaskManager.Task::(prepare) Task=`48f264e1-89a8-4986-a602-c160d48d4bd8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3302::DEBUG::2012-12-09 13:18:24,928::task::588::TaskManager.Task::(_updateState) Task=`48f264e1-89a8-4986-a602-c160d48d4bd8`::moving from state preparing -> state finished
>Thread-3302::DEBUG::2012-12-09 13:18:24,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3302::DEBUG::2012-12-09 13:18:24,928::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3302::DEBUG::2012-12-09 13:18:24,928::task::978::TaskManager.Task::(_decref) Task=`48f264e1-89a8-4986-a602-c160d48d4bd8`::ref 0 aborting False
>Thread-3303::DEBUG::2012-12-09 13:18:24,949::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3303::DEBUG::2012-12-09 13:18:24,950::task::588::TaskManager.Task::(_updateState) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::moving from state init -> state preparing
>Thread-3303::INFO::2012-12-09 13:18:24,951::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3303::DEBUG::2012-12-09 13:18:24,951::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ac8d1d55-e718-4616-865a-3471bbb46bee`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3303::DEBUG::2012-12-09 13:18:24,952::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3303::DEBUG::2012-12-09 13:18:24,952::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3303::DEBUG::2012-12-09 13:18:24,953::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ac8d1d55-e718-4616-865a-3471bbb46bee`::Granted request
>Thread-3303::DEBUG::2012-12-09 13:18:24,955::task::817::TaskManager.Task::(resourceAcquired) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3303::DEBUG::2012-12-09 13:18:24,956::task::978::TaskManager.Task::(_decref) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::ref 1 aborting False
>Thread-3303::DEBUG::2012-12-09 13:18:24,957::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3303::DEBUG::2012-12-09 13:18:24,958::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:24,986::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0349618 s, 29.3 MB/s\n'; <rc> = 0
>Thread-3303::DEBUG::2012-12-09 13:18:25,056::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3303::WARNING::2012-12-09 13:18:25,057::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3303::DEBUG::2012-12-09 13:18:25,058::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3303::WARNING::2012-12-09 13:18:25,058::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3303::INFO::2012-12-09 13:18:25,058::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3303::DEBUG::2012-12-09 13:18:25,059::task::1172::TaskManager.Task::(prepare) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3303::DEBUG::2012-12-09 13:18:25,059::task::588::TaskManager.Task::(_updateState) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::moving from state preparing -> state finished
>Thread-3303::DEBUG::2012-12-09 13:18:25,059::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3303::DEBUG::2012-12-09 13:18:25,059::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3303::DEBUG::2012-12-09 13:18:25,060::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3303::DEBUG::2012-12-09 13:18:25,060::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3303::DEBUG::2012-12-09 13:18:25,060::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3303::DEBUG::2012-12-09 13:18:25,060::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3303::DEBUG::2012-12-09 13:18:25,060::task::978::TaskManager.Task::(_decref) Task=`22f8b3d1-3eb3-4b59-a9b9-122ffee2c96f`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:26,993::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:27,031::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016225 s, 63.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:18:28,778::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:28,779::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:28,877::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:28,878::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:28,878::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:28,879::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:18:29,038::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:29,079::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177792 s, 57.6 MB/s\n'; <rc> = 0
>Thread-3306::DEBUG::2012-12-09 13:18:30,519::task::588::TaskManager.Task::(_updateState) Task=`40753f25-31c4-4121-9622-d8e7db24458b`::moving from state init -> state preparing
>Thread-3306::INFO::2012-12-09 13:18:30,520::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3306::INFO::2012-12-09 13:18:30,520::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123100280762', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-3306::DEBUG::2012-12-09 13:18:30,520::task::1172::TaskManager.Task::(prepare) Task=`40753f25-31c4-4121-9622-d8e7db24458b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123100280762', 'lastCheck': '5.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-3306::DEBUG::2012-12-09 13:18:30,520::task::588::TaskManager.Task::(_updateState) Task=`40753f25-31c4-4121-9622-d8e7db24458b`::moving from state preparing -> state finished
>Thread-3306::DEBUG::2012-12-09 13:18:30,520::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3306::DEBUG::2012-12-09 13:18:30,521::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3306::DEBUG::2012-12-09 13:18:30,521::task::978::TaskManager.Task::(_decref) Task=`40753f25-31c4-4121-9622-d8e7db24458b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:31,086::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:31,126::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164831 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:33,133::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:33,178::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016017 s, 63.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:18:34,694::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:34,703::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000404217 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:35,185::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:35,227::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174234 s, 58.8 MB/s\n'; <rc> = 0
>Thread-3310::DEBUG::2012-12-09 13:18:35,785::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3310::DEBUG::2012-12-09 13:18:35,787::task::588::TaskManager.Task::(_updateState) Task=`92d363c0-0acd-4127-aab7-403e467b9444`::moving from state init -> state preparing
>Thread-3310::INFO::2012-12-09 13:18:35,787::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3310::INFO::2012-12-09 13:18:35,788::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3310::DEBUG::2012-12-09 13:18:35,788::task::1172::TaskManager.Task::(prepare) Task=`92d363c0-0acd-4127-aab7-403e467b9444`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3310::DEBUG::2012-12-09 13:18:35,789::task::588::TaskManager.Task::(_updateState) Task=`92d363c0-0acd-4127-aab7-403e467b9444`::moving from state preparing -> state finished
>Thread-3310::DEBUG::2012-12-09 13:18:35,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3310::DEBUG::2012-12-09 13:18:35,789::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3310::DEBUG::2012-12-09 13:18:35,790::task::978::TaskManager.Task::(_decref) Task=`92d363c0-0acd-4127-aab7-403e467b9444`::ref 0 aborting False
>Thread-3311::DEBUG::2012-12-09 13:18:35,810::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3311::DEBUG::2012-12-09 13:18:35,811::task::588::TaskManager.Task::(_updateState) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::moving from state init -> state preparing
>Thread-3311::INFO::2012-12-09 13:18:35,811::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3311::DEBUG::2012-12-09 13:18:35,812::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`48a2ed83-90f3-4b54-8e5a-fccc03395c8d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3311::DEBUG::2012-12-09 13:18:35,812::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3311::DEBUG::2012-12-09 13:18:35,813::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3311::DEBUG::2012-12-09 13:18:35,813::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`48a2ed83-90f3-4b54-8e5a-fccc03395c8d`::Granted request
>Thread-3311::DEBUG::2012-12-09 13:18:35,814::task::817::TaskManager.Task::(resourceAcquired) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3311::DEBUG::2012-12-09 13:18:35,814::task::978::TaskManager.Task::(_decref) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::ref 1 aborting False
>Thread-3311::DEBUG::2012-12-09 13:18:35,816::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3311::DEBUG::2012-12-09 13:18:35,817::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3311::DEBUG::2012-12-09 13:18:35,922::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3311::WARNING::2012-12-09 13:18:35,923::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3311::DEBUG::2012-12-09 13:18:35,923::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3311::WARNING::2012-12-09 13:18:35,924::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3311::INFO::2012-12-09 13:18:35,924::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3311::DEBUG::2012-12-09 13:18:35,924::task::1172::TaskManager.Task::(prepare) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3311::DEBUG::2012-12-09 13:18:35,924::task::588::TaskManager.Task::(_updateState) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::moving from state preparing -> state finished
>Thread-3311::DEBUG::2012-12-09 13:18:35,925::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3311::DEBUG::2012-12-09 13:18:35,925::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3311::DEBUG::2012-12-09 13:18:35,925::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3311::DEBUG::2012-12-09 13:18:35,925::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3311::DEBUG::2012-12-09 13:18:35,925::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3311::DEBUG::2012-12-09 13:18:35,926::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3311::DEBUG::2012-12-09 13:18:35,926::task::978::TaskManager.Task::(_decref) Task=`89cfb47a-fb96-4d59-ac99-8a612485d504`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:37,233::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:37,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180685 s, 56.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:18:38,880::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:38,880::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:38,986::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:38,987::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:38,987::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:38,988::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:18:39,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:39,320::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163022 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:41,327::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:41,389::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192882 s, 53.1 MB/s\n'; <rc> = 0
>Thread-3314::DEBUG::2012-12-09 13:18:41,743::task::588::TaskManager.Task::(_updateState) Task=`04e6ebd4-2e51-4cfd-8298-17a340c369cc`::moving from state init -> state preparing
>Thread-3314::INFO::2012-12-09 13:18:41,743::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3314::INFO::2012-12-09 13:18:41,744::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101978778839', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-3314::DEBUG::2012-12-09 13:18:41,744::task::1172::TaskManager.Task::(prepare) Task=`04e6ebd4-2e51-4cfd-8298-17a340c369cc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101978778839', 'lastCheck': '7.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-3314::DEBUG::2012-12-09 13:18:41,744::task::588::TaskManager.Task::(_updateState) Task=`04e6ebd4-2e51-4cfd-8298-17a340c369cc`::moving from state preparing -> state finished
>Thread-3314::DEBUG::2012-12-09 13:18:41,745::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3314::DEBUG::2012-12-09 13:18:41,745::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3314::DEBUG::2012-12-09 13:18:41,745::task::978::TaskManager.Task::(_decref) Task=`04e6ebd4-2e51-4cfd-8298-17a340c369cc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:43,395::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:43,440::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01856 s, 55.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:18:44,707::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:44,719::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000509892 s, 8.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:45,447::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:45,489::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017431 s, 58.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:47,494::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:47,533::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163966 s, 62.5 MB/s\n'; <rc> = 0
>Thread-3318::DEBUG::2012-12-09 13:18:48,135::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3318::DEBUG::2012-12-09 13:18:48,136::task::588::TaskManager.Task::(_updateState) Task=`54467b64-5279-40cc-aad3-6a85421c9a9b`::moving from state init -> state preparing
>Thread-3318::INFO::2012-12-09 13:18:48,136::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3318::INFO::2012-12-09 13:18:48,136::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3318::DEBUG::2012-12-09 13:18:48,136::task::1172::TaskManager.Task::(prepare) Task=`54467b64-5279-40cc-aad3-6a85421c9a9b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3318::DEBUG::2012-12-09 13:18:48,137::task::588::TaskManager.Task::(_updateState) Task=`54467b64-5279-40cc-aad3-6a85421c9a9b`::moving from state preparing -> state finished
>Thread-3318::DEBUG::2012-12-09 13:18:48,137::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3318::DEBUG::2012-12-09 13:18:48,137::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3318::DEBUG::2012-12-09 13:18:48,137::task::978::TaskManager.Task::(_decref) Task=`54467b64-5279-40cc-aad3-6a85421c9a9b`::ref 0 aborting False
>Thread-3319::DEBUG::2012-12-09 13:18:48,163::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3319::DEBUG::2012-12-09 13:18:48,164::task::588::TaskManager.Task::(_updateState) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::moving from state init -> state preparing
>Thread-3319::INFO::2012-12-09 13:18:48,164::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3319::DEBUG::2012-12-09 13:18:48,164::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aba58579-82c3-493e-b174-0f180a3e8b18`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3319::DEBUG::2012-12-09 13:18:48,165::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3319::DEBUG::2012-12-09 13:18:48,165::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3319::DEBUG::2012-12-09 13:18:48,165::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`aba58579-82c3-493e-b174-0f180a3e8b18`::Granted request
>Thread-3319::DEBUG::2012-12-09 13:18:48,165::task::817::TaskManager.Task::(resourceAcquired) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3319::DEBUG::2012-12-09 13:18:48,166::task::978::TaskManager.Task::(_decref) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::ref 1 aborting False
>Thread-3319::DEBUG::2012-12-09 13:18:48,166::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3319::DEBUG::2012-12-09 13:18:48,167::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3319::DEBUG::2012-12-09 13:18:48,271::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3319::WARNING::2012-12-09 13:18:48,273::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3319::DEBUG::2012-12-09 13:18:48,273::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3319::WARNING::2012-12-09 13:18:48,273::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3319::INFO::2012-12-09 13:18:48,274::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3319::DEBUG::2012-12-09 13:18:48,274::task::1172::TaskManager.Task::(prepare) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3319::DEBUG::2012-12-09 13:18:48,274::task::588::TaskManager.Task::(_updateState) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::moving from state preparing -> state finished
>Thread-3319::DEBUG::2012-12-09 13:18:48,274::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3319::DEBUG::2012-12-09 13:18:48,275::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3319::DEBUG::2012-12-09 13:18:48,275::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3319::DEBUG::2012-12-09 13:18:48,275::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3319::DEBUG::2012-12-09 13:18:48,275::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3319::DEBUG::2012-12-09 13:18:48,275::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3319::DEBUG::2012-12-09 13:18:48,276::task::978::TaskManager.Task::(_decref) Task=`bbf6cb21-7049-4fea-82fb-eab757f2cded`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:18:48,988::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:48,989::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:49,088::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:49,089::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:49,090::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:49,090::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:18:49,539::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:49,585::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202507 s, 50.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:51,592::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:51,633::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166757 s, 61.4 MB/s\n'; <rc> = 0
>Thread-3322::DEBUG::2012-12-09 13:18:52,414::task::588::TaskManager.Task::(_updateState) Task=`8042b60d-1369-4338-b2b9-615d036f8816`::moving from state init -> state preparing
>Thread-3322::INFO::2012-12-09 13:18:52,415::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3322::INFO::2012-12-09 13:18:52,415::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127730369568', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.3', 'code': 358, 'valid': False}}
>Thread-3322::DEBUG::2012-12-09 13:18:52,415::task::1172::TaskManager.Task::(prepare) Task=`8042b60d-1369-4338-b2b9-615d036f8816`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0127730369568', 'lastCheck': '7.7', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.3', 'code': 358, 'valid': False}}
>Thread-3322::DEBUG::2012-12-09 13:18:52,416::task::588::TaskManager.Task::(_updateState) Task=`8042b60d-1369-4338-b2b9-615d036f8816`::moving from state preparing -> state finished
>Thread-3322::DEBUG::2012-12-09 13:18:52,416::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3322::DEBUG::2012-12-09 13:18:52,416::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3322::DEBUG::2012-12-09 13:18:52,416::task::978::TaskManager.Task::(_decref) Task=`8042b60d-1369-4338-b2b9-615d036f8816`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:53,640::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:53,681::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173916 s, 58.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:18:54,722::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:18:54,734::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00046802 s, 8.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:55,687::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:55,726::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169888 s, 60.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:18:57,733::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:57,777::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179186 s, 57.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:18:59,091::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:18:59,091::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:18:59,195::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:18:59,196::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:18:59,197::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:18:59,197::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3327::DEBUG::2012-12-09 13:18:59,342::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3327::DEBUG::2012-12-09 13:18:59,343::task::588::TaskManager.Task::(_updateState) Task=`52603998-f380-4e60-9a0e-ba580f20d5d5`::moving from state init -> state preparing
>Thread-3327::INFO::2012-12-09 13:18:59,343::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3327::INFO::2012-12-09 13:18:59,344::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3327::DEBUG::2012-12-09 13:18:59,344::task::1172::TaskManager.Task::(prepare) Task=`52603998-f380-4e60-9a0e-ba580f20d5d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3327::DEBUG::2012-12-09 13:18:59,344::task::588::TaskManager.Task::(_updateState) Task=`52603998-f380-4e60-9a0e-ba580f20d5d5`::moving from state preparing -> state finished
>Thread-3327::DEBUG::2012-12-09 13:18:59,344::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3327::DEBUG::2012-12-09 13:18:59,344::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3327::DEBUG::2012-12-09 13:18:59,345::task::978::TaskManager.Task::(_decref) Task=`52603998-f380-4e60-9a0e-ba580f20d5d5`::ref 0 aborting False
>Thread-3328::DEBUG::2012-12-09 13:18:59,357::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3328::DEBUG::2012-12-09 13:18:59,357::task::588::TaskManager.Task::(_updateState) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::moving from state init -> state preparing
>Thread-3328::INFO::2012-12-09 13:18:59,358::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3328::DEBUG::2012-12-09 13:18:59,358::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`268b9c9e-ad91-42b9-8069-a3f7f38a3ba7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3328::DEBUG::2012-12-09 13:18:59,358::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3328::DEBUG::2012-12-09 13:18:59,358::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3328::DEBUG::2012-12-09 13:18:59,359::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`268b9c9e-ad91-42b9-8069-a3f7f38a3ba7`::Granted request
>Thread-3328::DEBUG::2012-12-09 13:18:59,359::task::817::TaskManager.Task::(resourceAcquired) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3328::DEBUG::2012-12-09 13:18:59,359::task::978::TaskManager.Task::(_decref) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::ref 1 aborting False
>Thread-3328::DEBUG::2012-12-09 13:18:59,360::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3328::DEBUG::2012-12-09 13:18:59,360::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3328::DEBUG::2012-12-09 13:18:59,458::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3328::WARNING::2012-12-09 13:18:59,460::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3328::DEBUG::2012-12-09 13:18:59,460::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3328::WARNING::2012-12-09 13:18:59,460::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3328::INFO::2012-12-09 13:18:59,461::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3328::DEBUG::2012-12-09 13:18:59,461::task::1172::TaskManager.Task::(prepare) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3328::DEBUG::2012-12-09 13:18:59,461::task::588::TaskManager.Task::(_updateState) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::moving from state preparing -> state finished
>Thread-3328::DEBUG::2012-12-09 13:18:59,461::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3328::DEBUG::2012-12-09 13:18:59,462::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3328::DEBUG::2012-12-09 13:18:59,462::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3328::DEBUG::2012-12-09 13:18:59,462::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3328::DEBUG::2012-12-09 13:18:59,462::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3328::DEBUG::2012-12-09 13:18:59,462::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3328::DEBUG::2012-12-09 13:18:59,463::task::978::TaskManager.Task::(_decref) Task=`e8613ca1-67ed-4a0b-a3c8-1a178343b88a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:18:59,784::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:18:59,824::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185474 s, 55.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:01,831::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:01,871::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016049 s, 63.8 MB/s\n'; <rc> = 0
>Thread-3330::DEBUG::2012-12-09 13:19:03,371::task::588::TaskManager.Task::(_updateState) Task=`eaee99f0-6e24-4e95-b33e-b40b2b2c8ade`::moving from state init -> state preparing
>Thread-3330::INFO::2012-12-09 13:19:03,371::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3330::INFO::2012-12-09 13:19:03,372::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123069286346', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-3330::DEBUG::2012-12-09 13:19:03,372::task::1172::TaskManager.Task::(prepare) Task=`eaee99f0-6e24-4e95-b33e-b40b2b2c8ade`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0123069286346', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-3330::DEBUG::2012-12-09 13:19:03,372::task::588::TaskManager.Task::(_updateState) Task=`eaee99f0-6e24-4e95-b33e-b40b2b2c8ade`::moving from state preparing -> state finished
>Thread-3330::DEBUG::2012-12-09 13:19:03,372::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3330::DEBUG::2012-12-09 13:19:03,373::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3330::DEBUG::2012-12-09 13:19:03,373::task::978::TaskManager.Task::(_decref) Task=`eaee99f0-6e24-4e95-b33e-b40b2b2c8ade`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:03,878::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:03,925::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0197536 s, 51.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:04,737::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:04,745::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000435265 s, 9.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:05,931::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:05,972::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171627 s, 59.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:07,978::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:08,018::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163671 s, 62.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:09,198::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:09,199::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:09,293::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:09,295::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:09,295::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:09,295::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:19:10,025::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3334::DEBUG::2012-12-09 13:19:10,041::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3334::DEBUG::2012-12-09 13:19:10,043::task::588::TaskManager.Task::(_updateState) Task=`5e95209e-b35c-48e1-a8e5-2cfeb8c85222`::moving from state init -> state preparing
>Thread-3334::INFO::2012-12-09 13:19:10,043::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3334::INFO::2012-12-09 13:19:10,044::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3334::DEBUG::2012-12-09 13:19:10,045::task::1172::TaskManager.Task::(prepare) Task=`5e95209e-b35c-48e1-a8e5-2cfeb8c85222`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3334::DEBUG::2012-12-09 13:19:10,045::task::588::TaskManager.Task::(_updateState) Task=`5e95209e-b35c-48e1-a8e5-2cfeb8c85222`::moving from state preparing -> state finished
>Thread-3334::DEBUG::2012-12-09 13:19:10,046::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3334::DEBUG::2012-12-09 13:19:10,046::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3334::DEBUG::2012-12-09 13:19:10,047::task::978::TaskManager.Task::(_decref) Task=`5e95209e-b35c-48e1-a8e5-2cfeb8c85222`::ref 0 aborting False
>Thread-3335::DEBUG::2012-12-09 13:19:10,063::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3335::DEBUG::2012-12-09 13:19:10,064::task::588::TaskManager.Task::(_updateState) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::moving from state init -> state preparing
>Thread-3335::INFO::2012-12-09 13:19:10,064::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3335::DEBUG::2012-12-09 13:19:10,065::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e832e06c-a689-45ec-bc41-3a6d3dddf8eb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3335::DEBUG::2012-12-09 13:19:10,066::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3335::DEBUG::2012-12-09 13:19:10,067::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3335::DEBUG::2012-12-09 13:19:10,067::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`e832e06c-a689-45ec-bc41-3a6d3dddf8eb`::Granted request
>Thread-3335::DEBUG::2012-12-09 13:19:10,069::task::817::TaskManager.Task::(resourceAcquired) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3335::DEBUG::2012-12-09 13:19:10,069::task::978::TaskManager.Task::(_decref) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::ref 1 aborting False
>Thread-3335::DEBUG::2012-12-09 13:19:10,071::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3335::DEBUG::2012-12-09 13:19:10,072::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:10,094::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279997 s, 36.6 MB/s\n'; <rc> = 0
>Thread-3335::DEBUG::2012-12-09 13:19:10,168::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3335::WARNING::2012-12-09 13:19:10,169::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3335::DEBUG::2012-12-09 13:19:10,170::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3335::WARNING::2012-12-09 13:19:10,170::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3335::INFO::2012-12-09 13:19:10,170::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3335::DEBUG::2012-12-09 13:19:10,171::task::1172::TaskManager.Task::(prepare) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3335::DEBUG::2012-12-09 13:19:10,171::task::588::TaskManager.Task::(_updateState) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::moving from state preparing -> state finished
>Thread-3335::DEBUG::2012-12-09 13:19:10,171::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3335::DEBUG::2012-12-09 13:19:10,171::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3335::DEBUG::2012-12-09 13:19:10,172::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3335::DEBUG::2012-12-09 13:19:10,172::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3335::DEBUG::2012-12-09 13:19:10,172::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3335::DEBUG::2012-12-09 13:19:10,172::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3335::DEBUG::2012-12-09 13:19:10,172::task::978::TaskManager.Task::(_decref) Task=`e3743c41-d4aa-4076-aa27-0b0627752ba6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:12,101::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:12,142::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167025 s, 61.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:14,149::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:14,189::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164893 s, 62.1 MB/s\n'; <rc> = 0
>Thread-3338::DEBUG::2012-12-09 13:19:14,503::task::588::TaskManager.Task::(_updateState) Task=`85ebac4a-9c83-468d-bfa6-d11595ff8e39`::moving from state init -> state preparing
>Thread-3338::INFO::2012-12-09 13:19:14,504::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3338::INFO::2012-12-09 13:19:14,504::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00832295417786', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-3338::DEBUG::2012-12-09 13:19:14,504::task::1172::TaskManager.Task::(prepare) Task=`85ebac4a-9c83-468d-bfa6-d11595ff8e39`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00832295417786', 'lastCheck': '9.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.2', 'code': 358, 'valid': False}}
>Thread-3338::DEBUG::2012-12-09 13:19:14,504::task::588::TaskManager.Task::(_updateState) Task=`85ebac4a-9c83-468d-bfa6-d11595ff8e39`::moving from state preparing -> state finished
>Thread-3338::DEBUG::2012-12-09 13:19:14,504::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3338::DEBUG::2012-12-09 13:19:14,505::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3338::DEBUG::2012-12-09 13:19:14,505::task::978::TaskManager.Task::(_decref) Task=`85ebac4a-9c83-468d-bfa6-d11595ff8e39`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:19:14,748::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 13:19:14,748::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:19:14,749::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:14,759::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000409255 s, 5.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:14,759::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000409255 s, 5.0 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 13:19:14,760::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 13:19:14,760::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 13:19:14,760::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:19:14,760::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,761::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,761::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,761::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,761::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,762::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:14,858::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:14,859::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:19:14,859::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-50::WARNING::2012-12-09 13:19:14,860::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 13:19:14,860::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 13:19:14,860::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:19:14,861::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:14,962::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:14,963::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:14,971::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000414578 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:16,196::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:16,244::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179311 s, 57.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:18,252::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:18,302::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234376 s, 43.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:19,296::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:19,297::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:19,397::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:19,398::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:19,398::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:19,399::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:19:20,309::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:20,348::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016582 s, 61.8 MB/s\n'; <rc> = 0
>Thread-3343::DEBUG::2012-12-09 13:19:21,387::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3343::DEBUG::2012-12-09 13:19:21,387::task::588::TaskManager.Task::(_updateState) Task=`88dcd441-ced9-4750-adce-0d4fab0cc063`::moving from state init -> state preparing
>Thread-3343::INFO::2012-12-09 13:19:21,387::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3343::INFO::2012-12-09 13:19:21,388::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3343::DEBUG::2012-12-09 13:19:21,388::task::1172::TaskManager.Task::(prepare) Task=`88dcd441-ced9-4750-adce-0d4fab0cc063`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3343::DEBUG::2012-12-09 13:19:21,388::task::588::TaskManager.Task::(_updateState) Task=`88dcd441-ced9-4750-adce-0d4fab0cc063`::moving from state preparing -> state finished
>Thread-3343::DEBUG::2012-12-09 13:19:21,388::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3343::DEBUG::2012-12-09 13:19:21,389::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3343::DEBUG::2012-12-09 13:19:21,389::task::978::TaskManager.Task::(_decref) Task=`88dcd441-ced9-4750-adce-0d4fab0cc063`::ref 0 aborting False
>Thread-3344::DEBUG::2012-12-09 13:19:21,403::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3344::DEBUG::2012-12-09 13:19:21,404::task::588::TaskManager.Task::(_updateState) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::moving from state init -> state preparing
>Thread-3344::INFO::2012-12-09 13:19:21,404::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3344::DEBUG::2012-12-09 13:19:21,404::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bf3606d4-8b18-4d0e-b519-19ab941a3d38`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3344::DEBUG::2012-12-09 13:19:21,405::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3344::DEBUG::2012-12-09 13:19:21,405::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3344::DEBUG::2012-12-09 13:19:21,405::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`bf3606d4-8b18-4d0e-b519-19ab941a3d38`::Granted request
>Thread-3344::DEBUG::2012-12-09 13:19:21,406::task::817::TaskManager.Task::(resourceAcquired) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3344::DEBUG::2012-12-09 13:19:21,406::task::978::TaskManager.Task::(_decref) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::ref 1 aborting False
>Thread-3344::DEBUG::2012-12-09 13:19:21,407::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3344::DEBUG::2012-12-09 13:19:21,408::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3344::DEBUG::2012-12-09 13:19:21,510::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3344::WARNING::2012-12-09 13:19:21,511::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3344::DEBUG::2012-12-09 13:19:21,511::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3344::WARNING::2012-12-09 13:19:21,512::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3344::INFO::2012-12-09 13:19:21,512::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3344::DEBUG::2012-12-09 13:19:21,512::task::1172::TaskManager.Task::(prepare) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3344::DEBUG::2012-12-09 13:19:21,512::task::588::TaskManager.Task::(_updateState) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::moving from state preparing -> state finished
>Thread-3344::DEBUG::2012-12-09 13:19:21,513::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3344::DEBUG::2012-12-09 13:19:21,513::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3344::DEBUG::2012-12-09 13:19:21,513::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3344::DEBUG::2012-12-09 13:19:21,513::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3344::DEBUG::2012-12-09 13:19:21,514::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3344::DEBUG::2012-12-09 13:19:21,514::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3344::DEBUG::2012-12-09 13:19:21,514::task::978::TaskManager.Task::(_decref) Task=`a63cf7df-5c94-4eb3-8519-7dca0315003c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:22,355::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:22,406::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263854 s, 38.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:24,412::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:24,452::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167777 s, 61.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:24,974::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:24,986::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000483397 s, 8.5 MB/s\n'; <rc> = 0
>Thread-3346::DEBUG::2012-12-09 13:19:25,374::task::588::TaskManager.Task::(_updateState) Task=`4818be1f-a6f9-4dfc-a7a7-6ed37dfd8fcc`::moving from state init -> state preparing
>Thread-3346::INFO::2012-12-09 13:19:25,375::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3346::INFO::2012-12-09 13:19:25,375::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118389129639', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-3346::DEBUG::2012-12-09 13:19:25,375::task::1172::TaskManager.Task::(prepare) Task=`4818be1f-a6f9-4dfc-a7a7-6ed37dfd8fcc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118389129639', 'lastCheck': '0.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-3346::DEBUG::2012-12-09 13:19:25,375::task::588::TaskManager.Task::(_updateState) Task=`4818be1f-a6f9-4dfc-a7a7-6ed37dfd8fcc`::moving from state preparing -> state finished
>Thread-3346::DEBUG::2012-12-09 13:19:25,376::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3346::DEBUG::2012-12-09 13:19:25,376::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3346::DEBUG::2012-12-09 13:19:25,376::task::978::TaskManager.Task::(_decref) Task=`4818be1f-a6f9-4dfc-a7a7-6ed37dfd8fcc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:26,459::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:26,501::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179839 s, 56.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:28,508::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:28,547::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016525 s, 62.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:29,400::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:29,401::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:29,516::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:29,518::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:29,519::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:29,520::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:19:30,554::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:30,595::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159265 s, 64.3 MB/s\n'; <rc> = 0
>Thread-3351::DEBUG::2012-12-09 13:19:32,361::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3351::DEBUG::2012-12-09 13:19:32,361::task::588::TaskManager.Task::(_updateState) Task=`10b89d46-ae99-455a-8bcc-62c3aa31f1a4`::moving from state init -> state preparing
>Thread-3351::INFO::2012-12-09 13:19:32,361::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3351::INFO::2012-12-09 13:19:32,362::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3351::DEBUG::2012-12-09 13:19:32,362::task::1172::TaskManager.Task::(prepare) Task=`10b89d46-ae99-455a-8bcc-62c3aa31f1a4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3351::DEBUG::2012-12-09 13:19:32,362::task::588::TaskManager.Task::(_updateState) Task=`10b89d46-ae99-455a-8bcc-62c3aa31f1a4`::moving from state preparing -> state finished
>Thread-3351::DEBUG::2012-12-09 13:19:32,362::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3351::DEBUG::2012-12-09 13:19:32,363::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3351::DEBUG::2012-12-09 13:19:32,363::task::978::TaskManager.Task::(_decref) Task=`10b89d46-ae99-455a-8bcc-62c3aa31f1a4`::ref 0 aborting False
>Thread-3352::DEBUG::2012-12-09 13:19:32,375::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3352::DEBUG::2012-12-09 13:19:32,376::task::588::TaskManager.Task::(_updateState) Task=`eb23e542-890e-4925-b216-b1d860be180e`::moving from state init -> state preparing
>Thread-3352::INFO::2012-12-09 13:19:32,376::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3352::DEBUG::2012-12-09 13:19:32,376::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`46e71dde-b183-4143-8d45-e3d135a5b958`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3352::DEBUG::2012-12-09 13:19:32,377::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3352::DEBUG::2012-12-09 13:19:32,377::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3352::DEBUG::2012-12-09 13:19:32,377::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`46e71dde-b183-4143-8d45-e3d135a5b958`::Granted request
>Thread-3352::DEBUG::2012-12-09 13:19:32,377::task::817::TaskManager.Task::(resourceAcquired) Task=`eb23e542-890e-4925-b216-b1d860be180e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3352::DEBUG::2012-12-09 13:19:32,377::task::978::TaskManager.Task::(_decref) Task=`eb23e542-890e-4925-b216-b1d860be180e`::ref 1 aborting False
>Thread-3352::DEBUG::2012-12-09 13:19:32,378::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3352::DEBUG::2012-12-09 13:19:32,379::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3352::DEBUG::2012-12-09 13:19:32,480::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3352::WARNING::2012-12-09 13:19:32,481::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3352::DEBUG::2012-12-09 13:19:32,481::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3352::WARNING::2012-12-09 13:19:32,482::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3352::INFO::2012-12-09 13:19:32,482::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3352::DEBUG::2012-12-09 13:19:32,483::task::1172::TaskManager.Task::(prepare) Task=`eb23e542-890e-4925-b216-b1d860be180e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3352::DEBUG::2012-12-09 13:19:32,483::task::588::TaskManager.Task::(_updateState) Task=`eb23e542-890e-4925-b216-b1d860be180e`::moving from state preparing -> state finished
>Thread-3352::DEBUG::2012-12-09 13:19:32,483::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3352::DEBUG::2012-12-09 13:19:32,484::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3352::DEBUG::2012-12-09 13:19:32,484::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3352::DEBUG::2012-12-09 13:19:32,484::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3352::DEBUG::2012-12-09 13:19:32,484::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3352::DEBUG::2012-12-09 13:19:32,484::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3352::DEBUG::2012-12-09 13:19:32,485::task::978::TaskManager.Task::(_decref) Task=`eb23e542-890e-4925-b216-b1d860be180e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:32,601::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:32,659::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239138 s, 42.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:34,665::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:34,705::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171587 s, 59.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:34,989::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:35,000::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000399158 s, 10.3 MB/s\n'; <rc> = 0
>Thread-3354::DEBUG::2012-12-09 13:19:35,542::task::588::TaskManager.Task::(_updateState) Task=`a7a6f27c-ba32-4a39-8b8a-df2d96fa24dc`::moving from state init -> state preparing
>Thread-3354::INFO::2012-12-09 13:19:35,542::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3354::INFO::2012-12-09 13:19:35,543::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119330883026', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-3354::DEBUG::2012-12-09 13:19:35,543::task::1172::TaskManager.Task::(prepare) Task=`a7a6f27c-ba32-4a39-8b8a-df2d96fa24dc`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119330883026', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.0', 'code': 358, 'valid': False}}
>Thread-3354::DEBUG::2012-12-09 13:19:35,543::task::588::TaskManager.Task::(_updateState) Task=`a7a6f27c-ba32-4a39-8b8a-df2d96fa24dc`::moving from state preparing -> state finished
>Thread-3354::DEBUG::2012-12-09 13:19:35,543::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3354::DEBUG::2012-12-09 13:19:35,543::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3354::DEBUG::2012-12-09 13:19:35,544::task::978::TaskManager.Task::(_decref) Task=`a7a6f27c-ba32-4a39-8b8a-df2d96fa24dc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:36,712::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:36,752::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174533 s, 58.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:38,759::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:38,798::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160238 s, 63.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:39,521::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:39,522::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:39,621::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:39,622::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:39,622::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:39,623::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:19:40,805::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:40,851::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210171 s, 48.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:42,857::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:42,897::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165551 s, 61.9 MB/s\n'; <rc> = 0
>Thread-3359::DEBUG::2012-12-09 13:19:43,270::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3359::DEBUG::2012-12-09 13:19:43,271::task::588::TaskManager.Task::(_updateState) Task=`b284018f-c79f-480d-90a3-cba7194321ce`::moving from state init -> state preparing
>Thread-3359::INFO::2012-12-09 13:19:43,271::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3359::INFO::2012-12-09 13:19:43,271::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3359::DEBUG::2012-12-09 13:19:43,271::task::1172::TaskManager.Task::(prepare) Task=`b284018f-c79f-480d-90a3-cba7194321ce`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3359::DEBUG::2012-12-09 13:19:43,272::task::588::TaskManager.Task::(_updateState) Task=`b284018f-c79f-480d-90a3-cba7194321ce`::moving from state preparing -> state finished
>Thread-3359::DEBUG::2012-12-09 13:19:43,272::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3359::DEBUG::2012-12-09 13:19:43,272::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3359::DEBUG::2012-12-09 13:19:43,272::task::978::TaskManager.Task::(_decref) Task=`b284018f-c79f-480d-90a3-cba7194321ce`::ref 0 aborting False
>Thread-3360::DEBUG::2012-12-09 13:19:43,286::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3360::DEBUG::2012-12-09 13:19:43,287::task::588::TaskManager.Task::(_updateState) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::moving from state init -> state preparing
>Thread-3360::INFO::2012-12-09 13:19:43,287::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3360::DEBUG::2012-12-09 13:19:43,287::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1fce43c4-c897-4a20-b6ce-75c712a82597`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3360::DEBUG::2012-12-09 13:19:43,287::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3360::DEBUG::2012-12-09 13:19:43,288::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3360::DEBUG::2012-12-09 13:19:43,288::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`1fce43c4-c897-4a20-b6ce-75c712a82597`::Granted request
>Thread-3360::DEBUG::2012-12-09 13:19:43,288::task::817::TaskManager.Task::(resourceAcquired) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3360::DEBUG::2012-12-09 13:19:43,288::task::978::TaskManager.Task::(_decref) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::ref 1 aborting False
>Thread-3360::DEBUG::2012-12-09 13:19:43,289::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3360::DEBUG::2012-12-09 13:19:43,289::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3360::DEBUG::2012-12-09 13:19:43,389::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3360::WARNING::2012-12-09 13:19:43,390::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3360::DEBUG::2012-12-09 13:19:43,391::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3360::WARNING::2012-12-09 13:19:43,391::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3360::INFO::2012-12-09 13:19:43,392::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3360::DEBUG::2012-12-09 13:19:43,392::task::1172::TaskManager.Task::(prepare) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3360::DEBUG::2012-12-09 13:19:43,392::task::588::TaskManager.Task::(_updateState) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::moving from state preparing -> state finished
>Thread-3360::DEBUG::2012-12-09 13:19:43,392::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3360::DEBUG::2012-12-09 13:19:43,392::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3360::DEBUG::2012-12-09 13:19:43,393::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3360::DEBUG::2012-12-09 13:19:43,393::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3360::DEBUG::2012-12-09 13:19:43,393::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3360::DEBUG::2012-12-09 13:19:43,393::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3360::DEBUG::2012-12-09 13:19:43,394::task::978::TaskManager.Task::(_decref) Task=`c356c360-60ee-4309-a383-16bf0d1c4b41`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:44,904::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:44,951::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212327 s, 48.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:19:45,003::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:45,011::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00049087 s, 8.3 MB/s\n'; <rc> = 0
>Thread-3362::DEBUG::2012-12-09 13:19:46,146::task::588::TaskManager.Task::(_updateState) Task=`fed539c9-79b3-4def-8e0c-ececa18d8529`::moving from state init -> state preparing
>Thread-3362::INFO::2012-12-09 13:19:46,147::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3362::INFO::2012-12-09 13:19:46,147::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00808310508728', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3362::DEBUG::2012-12-09 13:19:46,147::task::1172::TaskManager.Task::(prepare) Task=`fed539c9-79b3-4def-8e0c-ececa18d8529`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00808310508728', 'lastCheck': '1.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3362::DEBUG::2012-12-09 13:19:46,147::task::588::TaskManager.Task::(_updateState) Task=`fed539c9-79b3-4def-8e0c-ececa18d8529`::moving from state preparing -> state finished
>Thread-3362::DEBUG::2012-12-09 13:19:46,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3362::DEBUG::2012-12-09 13:19:46,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3362::DEBUG::2012-12-09 13:19:46,148::task::978::TaskManager.Task::(_decref) Task=`fed539c9-79b3-4def-8e0c-ececa18d8529`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:46,958::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:46,999::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174503 s, 58.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:49,005::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:49,052::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219513 s, 46.6 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:49,624::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:19:49,624::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:49,625::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:49,724::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:49,725::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:49,725::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:49,726::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:19:51,058::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:51,107::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243098 s, 42.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:53,114::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:53,155::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176319 s, 58.1 MB/s\n'; <rc> = 0
>Thread-3367::DEBUG::2012-12-09 13:19:54,459::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3367::DEBUG::2012-12-09 13:19:54,460::task::588::TaskManager.Task::(_updateState) Task=`b7c0a686-a55b-4da4-a0ee-780295456dd1`::moving from state init -> state preparing
>Thread-3367::INFO::2012-12-09 13:19:54,461::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3367::INFO::2012-12-09 13:19:54,461::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3367::DEBUG::2012-12-09 13:19:54,462::task::1172::TaskManager.Task::(prepare) Task=`b7c0a686-a55b-4da4-a0ee-780295456dd1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3367::DEBUG::2012-12-09 13:19:54,462::task::588::TaskManager.Task::(_updateState) Task=`b7c0a686-a55b-4da4-a0ee-780295456dd1`::moving from state preparing -> state finished
>Thread-3367::DEBUG::2012-12-09 13:19:54,463::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3367::DEBUG::2012-12-09 13:19:54,463::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3367::DEBUG::2012-12-09 13:19:54,464::task::978::TaskManager.Task::(_decref) Task=`b7c0a686-a55b-4da4-a0ee-780295456dd1`::ref 0 aborting False
>Thread-3368::DEBUG::2012-12-09 13:19:54,483::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3368::DEBUG::2012-12-09 13:19:54,484::task::588::TaskManager.Task::(_updateState) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::moving from state init -> state preparing
>Thread-3368::INFO::2012-12-09 13:19:54,485::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3368::DEBUG::2012-12-09 13:19:54,485::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`35ce9cb3-a2a8-4e90-96df-94c792315fcc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3368::DEBUG::2012-12-09 13:19:54,486::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3368::DEBUG::2012-12-09 13:19:54,486::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3368::DEBUG::2012-12-09 13:19:54,487::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`35ce9cb3-a2a8-4e90-96df-94c792315fcc`::Granted request
>Thread-3368::DEBUG::2012-12-09 13:19:54,487::task::817::TaskManager.Task::(resourceAcquired) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3368::DEBUG::2012-12-09 13:19:54,488::task::978::TaskManager.Task::(_decref) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::ref 1 aborting False
>Thread-3368::DEBUG::2012-12-09 13:19:54,489::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3368::DEBUG::2012-12-09 13:19:54,490::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3368::DEBUG::2012-12-09 13:19:54,591::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3368::WARNING::2012-12-09 13:19:54,593::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3368::DEBUG::2012-12-09 13:19:54,593::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3368::WARNING::2012-12-09 13:19:54,594::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3368::INFO::2012-12-09 13:19:54,594::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3368::DEBUG::2012-12-09 13:19:54,594::task::1172::TaskManager.Task::(prepare) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3368::DEBUG::2012-12-09 13:19:54,594::task::588::TaskManager.Task::(_updateState) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::moving from state preparing -> state finished
>Thread-3368::DEBUG::2012-12-09 13:19:54,595::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3368::DEBUG::2012-12-09 13:19:54,595::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3368::DEBUG::2012-12-09 13:19:54,595::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3368::DEBUG::2012-12-09 13:19:54,595::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3368::DEBUG::2012-12-09 13:19:54,596::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3368::DEBUG::2012-12-09 13:19:54,596::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3368::DEBUG::2012-12-09 13:19:54,596::task::978::TaskManager.Task::(_decref) Task=`9e04a0d2-2a51-4716-8db3-bef6e4c9f16e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:19:55,013::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:19:55,025::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000603405 s, 6.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:55,162::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:55,199::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160053 s, 64.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:19:57,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:57,245::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159588 s, 64.2 MB/s\n'; <rc> = 0
>Thread-3370::DEBUG::2012-12-09 13:19:58,102::task::588::TaskManager.Task::(_updateState) Task=`bed76de4-7d37-4620-b908-718785a63e6b`::moving from state init -> state preparing
>Thread-3370::INFO::2012-12-09 13:19:58,103::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3370::INFO::2012-12-09 13:19:58,103::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118432044983', 'lastCheck': '3.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-3370::DEBUG::2012-12-09 13:19:58,103::task::1172::TaskManager.Task::(prepare) Task=`bed76de4-7d37-4620-b908-718785a63e6b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118432044983', 'lastCheck': '3.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.4', 'code': 358, 'valid': False}}
>Thread-3370::DEBUG::2012-12-09 13:19:58,103::task::588::TaskManager.Task::(_updateState) Task=`bed76de4-7d37-4620-b908-718785a63e6b`::moving from state preparing -> state finished
>Thread-3370::DEBUG::2012-12-09 13:19:58,103::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3370::DEBUG::2012-12-09 13:19:58,104::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3370::DEBUG::2012-12-09 13:19:58,104::task::978::TaskManager.Task::(_decref) Task=`bed76de4-7d37-4620-b908-718785a63e6b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:19:59,252::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:19:59,293::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176664 s, 58.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:19:59,727::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:19:59,728::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:19:59,828::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:19:59,829::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:19:59,830::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:19:59,830::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:20:01,299::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:01,338::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161418 s, 63.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:03,345::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:03,386::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170702 s, 60.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:20:05,028::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:05,036::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000387815 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:05,392::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:05,434::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163345 s, 62.7 MB/s\n'; <rc> = 0
>Thread-3374::DEBUG::2012-12-09 13:20:05,678::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3374::DEBUG::2012-12-09 13:20:05,678::task::588::TaskManager.Task::(_updateState) Task=`eb8f5e36-389e-41a2-81a6-d9d2edf30d62`::moving from state init -> state preparing
>Thread-3374::INFO::2012-12-09 13:20:05,678::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3374::INFO::2012-12-09 13:20:05,679::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3374::DEBUG::2012-12-09 13:20:05,679::task::1172::TaskManager.Task::(prepare) Task=`eb8f5e36-389e-41a2-81a6-d9d2edf30d62`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3374::DEBUG::2012-12-09 13:20:05,679::task::588::TaskManager.Task::(_updateState) Task=`eb8f5e36-389e-41a2-81a6-d9d2edf30d62`::moving from state preparing -> state finished
>Thread-3374::DEBUG::2012-12-09 13:20:05,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3374::DEBUG::2012-12-09 13:20:05,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3374::DEBUG::2012-12-09 13:20:05,680::task::978::TaskManager.Task::(_decref) Task=`eb8f5e36-389e-41a2-81a6-d9d2edf30d62`::ref 0 aborting False
>Thread-3375::DEBUG::2012-12-09 13:20:05,692::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3375::DEBUG::2012-12-09 13:20:05,692::task::588::TaskManager.Task::(_updateState) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::moving from state init -> state preparing
>Thread-3375::INFO::2012-12-09 13:20:05,692::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3375::DEBUG::2012-12-09 13:20:05,693::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`203598e4-e220-4e1b-ba7c-428077d98e52`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3375::DEBUG::2012-12-09 13:20:05,693::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3375::DEBUG::2012-12-09 13:20:05,693::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3375::DEBUG::2012-12-09 13:20:05,693::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`203598e4-e220-4e1b-ba7c-428077d98e52`::Granted request
>Thread-3375::DEBUG::2012-12-09 13:20:05,694::task::817::TaskManager.Task::(resourceAcquired) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3375::DEBUG::2012-12-09 13:20:05,694::task::978::TaskManager.Task::(_decref) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::ref 1 aborting False
>Thread-3375::DEBUG::2012-12-09 13:20:05,695::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3375::DEBUG::2012-12-09 13:20:05,695::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3375::DEBUG::2012-12-09 13:20:05,793::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3375::WARNING::2012-12-09 13:20:05,794::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3375::DEBUG::2012-12-09 13:20:05,794::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3375::WARNING::2012-12-09 13:20:05,795::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3375::INFO::2012-12-09 13:20:05,795::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3375::DEBUG::2012-12-09 13:20:05,795::task::1172::TaskManager.Task::(prepare) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3375::DEBUG::2012-12-09 13:20:05,796::task::588::TaskManager.Task::(_updateState) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::moving from state preparing -> state finished
>Thread-3375::DEBUG::2012-12-09 13:20:05,796::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3375::DEBUG::2012-12-09 13:20:05,796::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3375::DEBUG::2012-12-09 13:20:05,796::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3375::DEBUG::2012-12-09 13:20:05,797::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3375::DEBUG::2012-12-09 13:20:05,797::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3375::DEBUG::2012-12-09 13:20:05,797::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3375::DEBUG::2012-12-09 13:20:05,797::task::978::TaskManager.Task::(_decref) Task=`3d79de37-5340-4c5f-8de5-d17dcd056a67`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:07,441::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:07,487::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210486 s, 48.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:09,493::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:09,533::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170587 s, 60.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:20:09,831::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:20:09,832::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:20:09,933::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:20:09,934::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:20:09,935::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:20:09,935::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3378::DEBUG::2012-12-09 13:20:10,921::task::588::TaskManager.Task::(_updateState) Task=`56ec550c-16aa-4dd5-93b9-4ac62eb56fa8`::moving from state init -> state preparing
>Thread-3378::INFO::2012-12-09 13:20:10,922::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3378::INFO::2012-12-09 13:20:10,922::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839900970459', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-3378::DEBUG::2012-12-09 13:20:10,922::task::1172::TaskManager.Task::(prepare) Task=`56ec550c-16aa-4dd5-93b9-4ac62eb56fa8`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00839900970459', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.0', 'code': 358, 'valid': False}}
>Thread-3378::DEBUG::2012-12-09 13:20:10,923::task::588::TaskManager.Task::(_updateState) Task=`56ec550c-16aa-4dd5-93b9-4ac62eb56fa8`::moving from state preparing -> state finished
>Thread-3378::DEBUG::2012-12-09 13:20:10,923::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3378::DEBUG::2012-12-09 13:20:10,923::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3378::DEBUG::2012-12-09 13:20:10,923::task::978::TaskManager.Task::(_decref) Task=`56ec550c-16aa-4dd5-93b9-4ac62eb56fa8`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:11,541::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:11,580::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171741 s, 59.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:13,587::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:13,629::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174453 s, 58.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:20:15,039::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:15,050::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000353825 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:15,636::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:15,686::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264055 s, 38.8 MB/s\n'; <rc> = 0
>Thread-3382::DEBUG::2012-12-09 13:20:16,664::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3382::DEBUG::2012-12-09 13:20:16,665::task::588::TaskManager.Task::(_updateState) Task=`be2700a0-4313-4c27-9701-6456260b082d`::moving from state init -> state preparing
>Thread-3382::INFO::2012-12-09 13:20:16,666::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3382::INFO::2012-12-09 13:20:16,666::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3382::DEBUG::2012-12-09 13:20:16,667::task::1172::TaskManager.Task::(prepare) Task=`be2700a0-4313-4c27-9701-6456260b082d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3382::DEBUG::2012-12-09 13:20:16,667::task::588::TaskManager.Task::(_updateState) Task=`be2700a0-4313-4c27-9701-6456260b082d`::moving from state preparing -> state finished
>Thread-3382::DEBUG::2012-12-09 13:20:16,667::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3382::DEBUG::2012-12-09 13:20:16,668::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3382::DEBUG::2012-12-09 13:20:16,668::task::978::TaskManager.Task::(_decref) Task=`be2700a0-4313-4c27-9701-6456260b082d`::ref 0 aborting False
>Thread-3383::DEBUG::2012-12-09 13:20:16,689::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3383::DEBUG::2012-12-09 13:20:16,690::task::588::TaskManager.Task::(_updateState) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::moving from state init -> state preparing
>Thread-3383::INFO::2012-12-09 13:20:16,690::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3383::DEBUG::2012-12-09 13:20:16,690::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`067d996b-9e74-4e2d-b40f-fe4c4761e6a0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3383::DEBUG::2012-12-09 13:20:16,690::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3383::DEBUG::2012-12-09 13:20:16,691::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3383::DEBUG::2012-12-09 13:20:16,691::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`067d996b-9e74-4e2d-b40f-fe4c4761e6a0`::Granted request
>Thread-3383::DEBUG::2012-12-09 13:20:16,691::task::817::TaskManager.Task::(resourceAcquired) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3383::DEBUG::2012-12-09 13:20:16,691::task::978::TaskManager.Task::(_decref) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::ref 1 aborting False
>Thread-3383::DEBUG::2012-12-09 13:20:16,692::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3383::DEBUG::2012-12-09 13:20:16,693::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3383::DEBUG::2012-12-09 13:20:16,792::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3383::WARNING::2012-12-09 13:20:16,793::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3383::DEBUG::2012-12-09 13:20:16,794::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3383::WARNING::2012-12-09 13:20:16,794::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3383::INFO::2012-12-09 13:20:16,794::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3383::DEBUG::2012-12-09 13:20:16,795::task::1172::TaskManager.Task::(prepare) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3383::DEBUG::2012-12-09 13:20:16,795::task::588::TaskManager.Task::(_updateState) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::moving from state preparing -> state finished
>Thread-3383::DEBUG::2012-12-09 13:20:16,795::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3383::DEBUG::2012-12-09 13:20:16,795::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3383::DEBUG::2012-12-09 13:20:16,796::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3383::DEBUG::2012-12-09 13:20:16,796::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3383::DEBUG::2012-12-09 13:20:16,796::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3383::DEBUG::2012-12-09 13:20:16,796::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3383::DEBUG::2012-12-09 13:20:16,796::task::978::TaskManager.Task::(_decref) Task=`eded3bdb-a188-444d-9c32-b0f597da3cf4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:17,693::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:17,732::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157174 s, 65.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:19,739::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:19,778::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158941 s, 64.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:20:19,936::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:20:19,937::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:20:20,037::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:20:20,038::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:20:20,038::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:20:20,038::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:20:21,785::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:21,826::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181983 s, 56.3 MB/s\n'; <rc> = 0
>Thread-3386::DEBUG::2012-12-09 13:20:22,549::task::588::TaskManager.Task::(_updateState) Task=`5dc871ac-bf1d-4f5b-b102-340f2ec87318`::moving from state init -> state preparing
>Thread-3386::INFO::2012-12-09 13:20:22,549::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3386::INFO::2012-12-09 13:20:22,550::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119869709015', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-3386::DEBUG::2012-12-09 13:20:22,550::task::1172::TaskManager.Task::(prepare) Task=`5dc871ac-bf1d-4f5b-b102-340f2ec87318`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119869709015', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.5', 'code': 358, 'valid': False}}
>Thread-3386::DEBUG::2012-12-09 13:20:22,551::task::588::TaskManager.Task::(_updateState) Task=`5dc871ac-bf1d-4f5b-b102-340f2ec87318`::moving from state preparing -> state finished
>Thread-3386::DEBUG::2012-12-09 13:20:22,551::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3386::DEBUG::2012-12-09 13:20:22,551::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3386::DEBUG::2012-12-09 13:20:22,552::task::978::TaskManager.Task::(_decref) Task=`5dc871ac-bf1d-4f5b-b102-340f2ec87318`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:23,833::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:23,874::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175501 s, 58.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:20:25,053::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:25,063::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000501525 s, 8.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:25,880::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:25,920::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168104 s, 60.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:27,927::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:27,967::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159143 s, 64.3 MB/s\n'; <rc> = 0
>Thread-3389::DEBUG::2012-12-09 13:20:28,086::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3389::DEBUG::2012-12-09 13:20:28,087::task::588::TaskManager.Task::(_updateState) Task=`95511ada-5c5d-446e-91c9-36aa8000138f`::moving from state init -> state preparing
>Thread-3389::INFO::2012-12-09 13:20:28,087::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3389::INFO::2012-12-09 13:20:28,088::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3389::DEBUG::2012-12-09 13:20:28,088::task::1172::TaskManager.Task::(prepare) Task=`95511ada-5c5d-446e-91c9-36aa8000138f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3389::DEBUG::2012-12-09 13:20:28,089::task::588::TaskManager.Task::(_updateState) Task=`95511ada-5c5d-446e-91c9-36aa8000138f`::moving from state preparing -> state finished
>Thread-3389::DEBUG::2012-12-09 13:20:28,089::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3389::DEBUG::2012-12-09 13:20:28,090::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3389::DEBUG::2012-12-09 13:20:28,090::task::978::TaskManager.Task::(_decref) Task=`95511ada-5c5d-446e-91c9-36aa8000138f`::ref 0 aborting False
>Thread-3390::DEBUG::2012-12-09 13:20:28,107::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3390::DEBUG::2012-12-09 13:20:28,108::task::588::TaskManager.Task::(_updateState) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::moving from state init -> state preparing
>Thread-3390::INFO::2012-12-09 13:20:28,108::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3390::DEBUG::2012-12-09 13:20:28,109::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8b7b69e5-2c42-4c75-8620-a9a75f4e1c0a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3390::DEBUG::2012-12-09 13:20:28,109::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3390::DEBUG::2012-12-09 13:20:28,110::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3390::DEBUG::2012-12-09 13:20:28,110::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`8b7b69e5-2c42-4c75-8620-a9a75f4e1c0a`::Granted request
>Thread-3390::DEBUG::2012-12-09 13:20:28,111::task::817::TaskManager.Task::(resourceAcquired) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3390::DEBUG::2012-12-09 13:20:28,111::task::978::TaskManager.Task::(_decref) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::ref 1 aborting False
>Thread-3390::DEBUG::2012-12-09 13:20:28,112::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3390::DEBUG::2012-12-09 13:20:28,113::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3390::DEBUG::2012-12-09 13:20:28,216::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3390::WARNING::2012-12-09 13:20:28,217::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3390::DEBUG::2012-12-09 13:20:28,217::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3390::WARNING::2012-12-09 13:20:28,218::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3390::INFO::2012-12-09 13:20:28,218::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3390::DEBUG::2012-12-09 13:20:28,218::task::1172::TaskManager.Task::(prepare) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3390::DEBUG::2012-12-09 13:20:28,219::task::588::TaskManager.Task::(_updateState) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::moving from state preparing -> state finished
>Thread-3390::DEBUG::2012-12-09 13:20:28,219::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3390::DEBUG::2012-12-09 13:20:28,219::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3390::DEBUG::2012-12-09 13:20:28,219::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3390::DEBUG::2012-12-09 13:20:28,219::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3390::DEBUG::2012-12-09 13:20:28,220::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3390::DEBUG::2012-12-09 13:20:28,220::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3390::DEBUG::2012-12-09 13:20:28,220::task::978::TaskManager.Task::(_decref) Task=`5d25311f-220a-47e0-8c82-bc39fe1a147e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:29,974::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:30,014::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163331 s, 62.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:20:30,039::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:20:30,041::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:20:30,138::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:20:30,139::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:20:30,140::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:20:30,140::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:20:32,021::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:32,062::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164931 s, 62.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:34,069::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:34,121::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185689 s, 55.1 MB/s\n'; <rc> = 0
>Thread-3394::DEBUG::2012-12-09 13:20:34,575::task::588::TaskManager.Task::(_updateState) Task=`66a45779-a5f8-4281-93fc-89ce5ca142b5`::moving from state init -> state preparing
>Thread-3394::INFO::2012-12-09 13:20:34,576::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3394::INFO::2012-12-09 13:20:34,576::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103030204773', 'lastCheck': '9.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-3394::DEBUG::2012-12-09 13:20:34,576::task::1172::TaskManager.Task::(prepare) Task=`66a45779-a5f8-4281-93fc-89ce5ca142b5`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0103030204773', 'lastCheck': '9.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-3394::DEBUG::2012-12-09 13:20:34,576::task::588::TaskManager.Task::(_updateState) Task=`66a45779-a5f8-4281-93fc-89ce5ca142b5`::moving from state preparing -> state finished
>Thread-3394::DEBUG::2012-12-09 13:20:34,576::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3394::DEBUG::2012-12-09 13:20:34,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3394::DEBUG::2012-12-09 13:20:34,577::task::978::TaskManager.Task::(_decref) Task=`66a45779-a5f8-4281-93fc-89ce5ca142b5`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:20:35,066::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:35,077::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000462441 s, 8.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:36,128::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:36,168::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158532 s, 64.6 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:38,175::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:38,221::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207411 s, 49.4 MB/s\n'; <rc> = 0
>Thread-3397::DEBUG::2012-12-09 13:20:38,677::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3397::DEBUG::2012-12-09 13:20:38,678::task::588::TaskManager.Task::(_updateState) Task=`711fdf96-f59e-4c20-82f4-f7b4db98a5e5`::moving from state init -> state preparing
>Thread-3397::INFO::2012-12-09 13:20:38,678::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3397::INFO::2012-12-09 13:20:38,679::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3397::DEBUG::2012-12-09 13:20:38,679::task::1172::TaskManager.Task::(prepare) Task=`711fdf96-f59e-4c20-82f4-f7b4db98a5e5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3397::DEBUG::2012-12-09 13:20:38,679::task::588::TaskManager.Task::(_updateState) Task=`711fdf96-f59e-4c20-82f4-f7b4db98a5e5`::moving from state preparing -> state finished
>Thread-3397::DEBUG::2012-12-09 13:20:38,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3397::DEBUG::2012-12-09 13:20:38,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3397::DEBUG::2012-12-09 13:20:38,680::task::978::TaskManager.Task::(_decref) Task=`711fdf96-f59e-4c20-82f4-f7b4db98a5e5`::ref 0 aborting False
>Thread-3398::DEBUG::2012-12-09 13:20:38,691::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3398::DEBUG::2012-12-09 13:20:38,692::task::588::TaskManager.Task::(_updateState) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::moving from state init -> state preparing
>Thread-3398::INFO::2012-12-09 13:20:38,692::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3398::DEBUG::2012-12-09 13:20:38,692::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`59a14c1d-575f-4406-8f98-a0c84b5ff3a6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3398::DEBUG::2012-12-09 13:20:38,692::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3398::DEBUG::2012-12-09 13:20:38,693::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3398::DEBUG::2012-12-09 13:20:38,693::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`59a14c1d-575f-4406-8f98-a0c84b5ff3a6`::Granted request
>Thread-3398::DEBUG::2012-12-09 13:20:38,693::task::817::TaskManager.Task::(resourceAcquired) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3398::DEBUG::2012-12-09 13:20:38,693::task::978::TaskManager.Task::(_decref) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::ref 1 aborting False
>Thread-3398::DEBUG::2012-12-09 13:20:38,694::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3398::DEBUG::2012-12-09 13:20:38,695::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3398::DEBUG::2012-12-09 13:20:38,793::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3398::WARNING::2012-12-09 13:20:38,795::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3398::DEBUG::2012-12-09 13:20:38,795::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3398::WARNING::2012-12-09 13:20:38,796::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3398::INFO::2012-12-09 13:20:38,796::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3398::DEBUG::2012-12-09 13:20:38,797::task::1172::TaskManager.Task::(prepare) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3398::DEBUG::2012-12-09 13:20:38,797::task::588::TaskManager.Task::(_updateState) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::moving from state preparing -> state finished
>Thread-3398::DEBUG::2012-12-09 13:20:38,797::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3398::DEBUG::2012-12-09 13:20:38,798::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3398::DEBUG::2012-12-09 13:20:38,798::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3398::DEBUG::2012-12-09 13:20:38,798::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3398::DEBUG::2012-12-09 13:20:38,799::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3398::DEBUG::2012-12-09 13:20:38,799::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3398::DEBUG::2012-12-09 13:20:38,799::task::978::TaskManager.Task::(_decref) Task=`49be26a1-c91f-4897-ab62-2671fd686083`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:20:40,141::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:20:40,142::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:40,228::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:20:40,246::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:20:40,249::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:20:40,249::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:20:40,250::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:20:40,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0227596 s, 45.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:42,281::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:42,321::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162391 s, 63.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:44,328::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:44,378::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167464 s, 61.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:20:45,080::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:45,093::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000652515 s, 6.3 MB/s\n'; <rc> = 0
>Thread-3402::DEBUG::2012-12-09 13:20:45,584::task::588::TaskManager.Task::(_updateState) Task=`333792e7-040f-4ab2-9cf2-56f0b67c283e`::moving from state init -> state preparing
>Thread-3402::INFO::2012-12-09 13:20:45,584::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3402::INFO::2012-12-09 13:20:45,584::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138609409332', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-3402::DEBUG::2012-12-09 13:20:45,585::task::1172::TaskManager.Task::(prepare) Task=`333792e7-040f-4ab2-9cf2-56f0b67c283e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0138609409332', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.3', 'code': 358, 'valid': False}}
>Thread-3402::DEBUG::2012-12-09 13:20:45,585::task::588::TaskManager.Task::(_updateState) Task=`333792e7-040f-4ab2-9cf2-56f0b67c283e`::moving from state preparing -> state finished
>Thread-3402::DEBUG::2012-12-09 13:20:45,585::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3402::DEBUG::2012-12-09 13:20:45,585::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3402::DEBUG::2012-12-09 13:20:45,585::task::978::TaskManager.Task::(_decref) Task=`333792e7-040f-4ab2-9cf2-56f0b67c283e`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:46,385::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:46,424::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165247 s, 62.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:48,431::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:48,479::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254121 s, 40.3 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:20:50,252::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:20:50,253::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:20:50,360::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:20:50,361::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:20:50,362::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:20:50,363::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:20:50,486::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:50,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0200394 s, 51.1 MB/s\n'; <rc> = 0
>Thread-3405::DEBUG::2012-12-09 13:20:50,566::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3405::DEBUG::2012-12-09 13:20:50,567::task::588::TaskManager.Task::(_updateState) Task=`2dab85c1-410c-4ee6-aa2e-1432abb9d27b`::moving from state init -> state preparing
>Thread-3405::INFO::2012-12-09 13:20:50,567::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3405::INFO::2012-12-09 13:20:50,567::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3405::DEBUG::2012-12-09 13:20:50,567::task::1172::TaskManager.Task::(prepare) Task=`2dab85c1-410c-4ee6-aa2e-1432abb9d27b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3405::DEBUG::2012-12-09 13:20:50,568::task::588::TaskManager.Task::(_updateState) Task=`2dab85c1-410c-4ee6-aa2e-1432abb9d27b`::moving from state preparing -> state finished
>Thread-3405::DEBUG::2012-12-09 13:20:50,568::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3405::DEBUG::2012-12-09 13:20:50,568::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3405::DEBUG::2012-12-09 13:20:50,568::task::978::TaskManager.Task::(_decref) Task=`2dab85c1-410c-4ee6-aa2e-1432abb9d27b`::ref 0 aborting False
>Thread-3406::DEBUG::2012-12-09 13:20:50,583::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3406::DEBUG::2012-12-09 13:20:50,583::task::588::TaskManager.Task::(_updateState) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::moving from state init -> state preparing
>Thread-3406::INFO::2012-12-09 13:20:50,584::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3406::DEBUG::2012-12-09 13:20:50,584::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`60feb64e-1045-4745-89bd-aa8fc177da4b`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3406::DEBUG::2012-12-09 13:20:50,585::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3406::DEBUG::2012-12-09 13:20:50,585::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3406::DEBUG::2012-12-09 13:20:50,585::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`60feb64e-1045-4745-89bd-aa8fc177da4b`::Granted request
>Thread-3406::DEBUG::2012-12-09 13:20:50,586::task::817::TaskManager.Task::(resourceAcquired) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3406::DEBUG::2012-12-09 13:20:50,586::task::978::TaskManager.Task::(_decref) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::ref 1 aborting False
>Thread-3406::DEBUG::2012-12-09 13:20:50,587::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3406::DEBUG::2012-12-09 13:20:50,588::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3406::DEBUG::2012-12-09 13:20:50,682::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3406::WARNING::2012-12-09 13:20:50,683::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3406::DEBUG::2012-12-09 13:20:50,683::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3406::WARNING::2012-12-09 13:20:50,683::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3406::INFO::2012-12-09 13:20:50,684::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3406::DEBUG::2012-12-09 13:20:50,684::task::1172::TaskManager.Task::(prepare) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3406::DEBUG::2012-12-09 13:20:50,684::task::588::TaskManager.Task::(_updateState) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::moving from state preparing -> state finished
>Thread-3406::DEBUG::2012-12-09 13:20:50,684::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3406::DEBUG::2012-12-09 13:20:50,685::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3406::DEBUG::2012-12-09 13:20:50,685::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3406::DEBUG::2012-12-09 13:20:50,685::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3406::DEBUG::2012-12-09 13:20:50,685::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3406::DEBUG::2012-12-09 13:20:50,686::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3406::DEBUG::2012-12-09 13:20:50,686::task::978::TaskManager.Task::(_decref) Task=`168c4470-4654-41e0-933d-ecc1ca6b87ed`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:52,537::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:52,575::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152627 s, 67.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:54,582::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:54,622::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160831 s, 63.7 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:20:55,096::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:20:55,104::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000511826 s, 8.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:20:56,628::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:56,676::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244538 s, 41.9 MB/s\n'; <rc> = 0
>Thread-3410::DEBUG::2012-12-09 13:20:57,473::task::588::TaskManager.Task::(_updateState) Task=`64555ec8-c9bf-44de-a6e1-dcb62b2f8eb1`::moving from state init -> state preparing
>Thread-3410::INFO::2012-12-09 13:20:57,473::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3410::INFO::2012-12-09 13:20:57,473::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850391387939', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-3410::DEBUG::2012-12-09 13:20:57,473::task::1172::TaskManager.Task::(prepare) Task=`64555ec8-c9bf-44de-a6e1-dcb62b2f8eb1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00850391387939', 'lastCheck': '2.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.1', 'code': 358, 'valid': False}}
>Thread-3410::DEBUG::2012-12-09 13:20:57,474::task::588::TaskManager.Task::(_updateState) Task=`64555ec8-c9bf-44de-a6e1-dcb62b2f8eb1`::moving from state preparing -> state finished
>Thread-3410::DEBUG::2012-12-09 13:20:57,474::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3410::DEBUG::2012-12-09 13:20:57,474::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3410::DEBUG::2012-12-09 13:20:57,474::task::978::TaskManager.Task::(_decref) Task=`64555ec8-c9bf-44de-a6e1-dcb62b2f8eb1`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:20:58,683::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:20:58,721::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015165 s, 67.5 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:21:00,364::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:00,365::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:00,470::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:00,472::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:00,473::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:00,473::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:00,728::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:00,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170921 s, 59.9 MB/s\n'; <rc> = 0
>Thread-3414::DEBUG::2012-12-09 13:21:02,193::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3414::DEBUG::2012-12-09 13:21:02,194::task::588::TaskManager.Task::(_updateState) Task=`23cf7333-9808-4747-a8ed-995e776a5e56`::moving from state init -> state preparing
>Thread-3414::INFO::2012-12-09 13:21:02,194::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3414::INFO::2012-12-09 13:21:02,194::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3414::DEBUG::2012-12-09 13:21:02,195::task::1172::TaskManager.Task::(prepare) Task=`23cf7333-9808-4747-a8ed-995e776a5e56`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3414::DEBUG::2012-12-09 13:21:02,195::task::588::TaskManager.Task::(_updateState) Task=`23cf7333-9808-4747-a8ed-995e776a5e56`::moving from state preparing -> state finished
>Thread-3414::DEBUG::2012-12-09 13:21:02,195::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3414::DEBUG::2012-12-09 13:21:02,195::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3414::DEBUG::2012-12-09 13:21:02,195::task::978::TaskManager.Task::(_decref) Task=`23cf7333-9808-4747-a8ed-995e776a5e56`::ref 0 aborting False
>Thread-3415::DEBUG::2012-12-09 13:21:02,208::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3415::DEBUG::2012-12-09 13:21:02,208::task::588::TaskManager.Task::(_updateState) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::moving from state init -> state preparing
>Thread-3415::INFO::2012-12-09 13:21:02,209::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3415::DEBUG::2012-12-09 13:21:02,209::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2e19f501-43a7-4118-8789-dbe46498974c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3415::DEBUG::2012-12-09 13:21:02,209::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3415::DEBUG::2012-12-09 13:21:02,209::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3415::DEBUG::2012-12-09 13:21:02,210::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`2e19f501-43a7-4118-8789-dbe46498974c`::Granted request
>Thread-3415::DEBUG::2012-12-09 13:21:02,210::task::817::TaskManager.Task::(resourceAcquired) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3415::DEBUG::2012-12-09 13:21:02,210::task::978::TaskManager.Task::(_decref) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::ref 1 aborting False
>Thread-3415::DEBUG::2012-12-09 13:21:02,211::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3415::DEBUG::2012-12-09 13:21:02,211::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3415::DEBUG::2012-12-09 13:21:02,314::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3415::WARNING::2012-12-09 13:21:02,316::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3415::DEBUG::2012-12-09 13:21:02,317::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3415::WARNING::2012-12-09 13:21:02,318::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3415::INFO::2012-12-09 13:21:02,319::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3415::DEBUG::2012-12-09 13:21:02,319::task::1172::TaskManager.Task::(prepare) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3415::DEBUG::2012-12-09 13:21:02,320::task::588::TaskManager.Task::(_updateState) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::moving from state preparing -> state finished
>Thread-3415::DEBUG::2012-12-09 13:21:02,320::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3415::DEBUG::2012-12-09 13:21:02,321::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3415::DEBUG::2012-12-09 13:21:02,321::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3415::DEBUG::2012-12-09 13:21:02,322::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3415::DEBUG::2012-12-09 13:21:02,322::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3415::DEBUG::2012-12-09 13:21:02,322::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3415::DEBUG::2012-12-09 13:21:02,323::task::978::TaskManager.Task::(_decref) Task=`352c35df-5faa-4c2c-96f7-b812a096dd6c`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:02,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:02,814::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162751 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:04,821::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:04,866::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195489 s, 52.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:21:05,108::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:05,119::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058467 s, 7.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:06,873::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:06,914::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017591 s, 58.2 MB/s\n'; <rc> = 0
>Thread-3418::DEBUG::2012-12-09 13:21:07,959::task::588::TaskManager.Task::(_updateState) Task=`4bf9145b-7633-4ff5-83e1-544353d7d41b`::moving from state init -> state preparing
>Thread-3418::INFO::2012-12-09 13:21:07,960::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3418::INFO::2012-12-09 13:21:07,961::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119588375092', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.5', 'code': 358, 'valid': False}}
>Thread-3418::DEBUG::2012-12-09 13:21:07,961::task::1172::TaskManager.Task::(prepare) Task=`4bf9145b-7633-4ff5-83e1-544353d7d41b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0119588375092', 'lastCheck': '2.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.5', 'code': 358, 'valid': False}}
>Thread-3418::DEBUG::2012-12-09 13:21:07,962::task::588::TaskManager.Task::(_updateState) Task=`4bf9145b-7633-4ff5-83e1-544353d7d41b`::moving from state preparing -> state finished
>Thread-3418::DEBUG::2012-12-09 13:21:07,962::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3418::DEBUG::2012-12-09 13:21:07,962::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3418::DEBUG::2012-12-09 13:21:07,962::task::978::TaskManager.Task::(_decref) Task=`4bf9145b-7633-4ff5-83e1-544353d7d41b`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:08,920::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:08,960::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164853 s, 62.1 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:21:10,474::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:10,475::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:10,570::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:10,572::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:10,572::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:10,572::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:10,967::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:11,005::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158838 s, 64.5 MB/s\n'; <rc> = 0
>Thread-3422::DEBUG::2012-12-09 13:21:12,706::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3422::DEBUG::2012-12-09 13:21:12,707::task::588::TaskManager.Task::(_updateState) Task=`34e9e566-344a-4fd3-a88f-70ad67fde2c9`::moving from state init -> state preparing
>Thread-3422::INFO::2012-12-09 13:21:12,707::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3422::INFO::2012-12-09 13:21:12,708::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3422::DEBUG::2012-12-09 13:21:12,709::task::1172::TaskManager.Task::(prepare) Task=`34e9e566-344a-4fd3-a88f-70ad67fde2c9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3422::DEBUG::2012-12-09 13:21:12,709::task::588::TaskManager.Task::(_updateState) Task=`34e9e566-344a-4fd3-a88f-70ad67fde2c9`::moving from state preparing -> state finished
>Thread-3422::DEBUG::2012-12-09 13:21:12,709::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3422::DEBUG::2012-12-09 13:21:12,710::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3422::DEBUG::2012-12-09 13:21:12,710::task::978::TaskManager.Task::(_decref) Task=`34e9e566-344a-4fd3-a88f-70ad67fde2c9`::ref 0 aborting False
>Thread-3423::DEBUG::2012-12-09 13:21:12,724::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3423::DEBUG::2012-12-09 13:21:12,724::task::588::TaskManager.Task::(_updateState) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::moving from state init -> state preparing
>Thread-3423::INFO::2012-12-09 13:21:12,724::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3423::DEBUG::2012-12-09 13:21:12,725::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7f41e900-8d6d-4c26-a0ad-44db0fbdeb56`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3423::DEBUG::2012-12-09 13:21:12,725::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3423::DEBUG::2012-12-09 13:21:12,725::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3423::DEBUG::2012-12-09 13:21:12,725::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`7f41e900-8d6d-4c26-a0ad-44db0fbdeb56`::Granted request
>Thread-3423::DEBUG::2012-12-09 13:21:12,726::task::817::TaskManager.Task::(resourceAcquired) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3423::DEBUG::2012-12-09 13:21:12,726::task::978::TaskManager.Task::(_decref) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::ref 1 aborting False
>Thread-3423::DEBUG::2012-12-09 13:21:12,727::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3423::DEBUG::2012-12-09 13:21:12,727::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3423::DEBUG::2012-12-09 13:21:12,825::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3423::WARNING::2012-12-09 13:21:12,827::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3423::DEBUG::2012-12-09 13:21:12,827::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3423::WARNING::2012-12-09 13:21:12,828::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3423::INFO::2012-12-09 13:21:12,828::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3423::DEBUG::2012-12-09 13:21:12,829::task::1172::TaskManager.Task::(prepare) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3423::DEBUG::2012-12-09 13:21:12,829::task::588::TaskManager.Task::(_updateState) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::moving from state preparing -> state finished
>Thread-3423::DEBUG::2012-12-09 13:21:12,830::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3423::DEBUG::2012-12-09 13:21:12,830::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3423::DEBUG::2012-12-09 13:21:12,830::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3423::DEBUG::2012-12-09 13:21:12,830::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3423::DEBUG::2012-12-09 13:21:12,831::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3423::DEBUG::2012-12-09 13:21:12,831::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3423::DEBUG::2012-12-09 13:21:12,831::task::978::TaskManager.Task::(_decref) Task=`878573f3-8d8a-4d9b-ae0d-2d23a4d1e216`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:13,011::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:13,051::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016746 s, 61.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:15,058::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:15,109::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201607 s, 50.8 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:21:15,123::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:15,135::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000819372 s, 5.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:17,115::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:17,158::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187315 s, 54.7 MB/s\n'; <rc> = 0
>Thread-3426::DEBUG::2012-12-09 13:21:18,553::task::588::TaskManager.Task::(_updateState) Task=`195db7a6-697c-4ab8-8e6f-17095d897219`::moving from state init -> state preparing
>Thread-3426::INFO::2012-12-09 13:21:18,554::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3426::INFO::2012-12-09 13:21:18,554::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130620002747', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.0', 'code': 358, 'valid': False}}
>Thread-3426::DEBUG::2012-12-09 13:21:18,555::task::1172::TaskManager.Task::(prepare) Task=`195db7a6-697c-4ab8-8e6f-17095d897219`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0130620002747', 'lastCheck': '3.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.0', 'code': 358, 'valid': False}}
>Thread-3426::DEBUG::2012-12-09 13:21:18,555::task::588::TaskManager.Task::(_updateState) Task=`195db7a6-697c-4ab8-8e6f-17095d897219`::moving from state preparing -> state finished
>Thread-3426::DEBUG::2012-12-09 13:21:18,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3426::DEBUG::2012-12-09 13:21:18,556::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3426::DEBUG::2012-12-09 13:21:18,556::task::978::TaskManager.Task::(_decref) Task=`195db7a6-697c-4ab8-8e6f-17095d897219`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:19,164::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:19,216::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222652 s, 46.0 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:21:20,573::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:20,574::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:20,683::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:20,684::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:20,684::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:20,684::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:21,223::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:21,265::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173294 s, 59.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:23,271::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:23,311::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163423 s, 62.7 MB/s\n'; <rc> = 0
>Thread-3430::DEBUG::2012-12-09 13:21:23,962::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3430::DEBUG::2012-12-09 13:21:23,963::task::588::TaskManager.Task::(_updateState) Task=`1a59bc56-143f-497d-8cca-65390a3c3dd9`::moving from state init -> state preparing
>Thread-3430::INFO::2012-12-09 13:21:23,963::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3430::INFO::2012-12-09 13:21:23,964::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3430::DEBUG::2012-12-09 13:21:23,964::task::1172::TaskManager.Task::(prepare) Task=`1a59bc56-143f-497d-8cca-65390a3c3dd9`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3430::DEBUG::2012-12-09 13:21:23,965::task::588::TaskManager.Task::(_updateState) Task=`1a59bc56-143f-497d-8cca-65390a3c3dd9`::moving from state preparing -> state finished
>Thread-3430::DEBUG::2012-12-09 13:21:23,965::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3430::DEBUG::2012-12-09 13:21:23,966::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3430::DEBUG::2012-12-09 13:21:23,966::task::978::TaskManager.Task::(_decref) Task=`1a59bc56-143f-497d-8cca-65390a3c3dd9`::ref 0 aborting False
>Thread-3431::DEBUG::2012-12-09 13:21:23,978::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3431::DEBUG::2012-12-09 13:21:23,978::task::588::TaskManager.Task::(_updateState) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::moving from state init -> state preparing
>Thread-3431::INFO::2012-12-09 13:21:23,978::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3431::DEBUG::2012-12-09 13:21:23,979::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6ed7b102-53da-4650-b115-d4ae9248e214`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3431::DEBUG::2012-12-09 13:21:23,979::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3431::DEBUG::2012-12-09 13:21:23,979::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3431::DEBUG::2012-12-09 13:21:23,979::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6ed7b102-53da-4650-b115-d4ae9248e214`::Granted request
>Thread-3431::DEBUG::2012-12-09 13:21:23,980::task::817::TaskManager.Task::(resourceAcquired) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3431::DEBUG::2012-12-09 13:21:23,980::task::978::TaskManager.Task::(_decref) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::ref 1 aborting False
>Thread-3431::DEBUG::2012-12-09 13:21:23,981::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3431::DEBUG::2012-12-09 13:21:23,981::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3431::DEBUG::2012-12-09 13:21:24,080::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3431::WARNING::2012-12-09 13:21:24,082::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3431::DEBUG::2012-12-09 13:21:24,082::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3431::WARNING::2012-12-09 13:21:24,083::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3431::INFO::2012-12-09 13:21:24,083::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3431::DEBUG::2012-12-09 13:21:24,083::task::1172::TaskManager.Task::(prepare) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3431::DEBUG::2012-12-09 13:21:24,084::task::588::TaskManager.Task::(_updateState) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::moving from state preparing -> state finished
>Thread-3431::DEBUG::2012-12-09 13:21:24,084::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3431::DEBUG::2012-12-09 13:21:24,084::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3431::DEBUG::2012-12-09 13:21:24,084::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3431::DEBUG::2012-12-09 13:21:24,085::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3431::DEBUG::2012-12-09 13:21:24,085::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3431::DEBUG::2012-12-09 13:21:24,085::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3431::DEBUG::2012-12-09 13:21:24,085::task::978::TaskManager.Task::(_decref) Task=`4b480b56-6631-42a4-a84d-ca1d4956d30a`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:21:25,141::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:25,155::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000707197 s, 5.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:25,318::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:25,360::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196043 s, 52.2 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:27,368::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:27,413::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199075 s, 51.4 MB/s\n'; <rc> = 0
>Thread-3434::DEBUG::2012-12-09 13:21:29,318::task::588::TaskManager.Task::(_updateState) Task=`481107bd-afb3-4cc2-860a-703522974f53`::moving from state init -> state preparing
>Thread-3434::INFO::2012-12-09 13:21:29,318::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3434::INFO::2012-12-09 13:21:29,318::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145390033722', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-3434::DEBUG::2012-12-09 13:21:29,318::task::1172::TaskManager.Task::(prepare) Task=`481107bd-afb3-4cc2-860a-703522974f53`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0145390033722', 'lastCheck': '4.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.6', 'code': 358, 'valid': False}}
>Thread-3434::DEBUG::2012-12-09 13:21:29,319::task::588::TaskManager.Task::(_updateState) Task=`481107bd-afb3-4cc2-860a-703522974f53`::moving from state preparing -> state finished
>Thread-3434::DEBUG::2012-12-09 13:21:29,319::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3434::DEBUG::2012-12-09 13:21:29,319::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3434::DEBUG::2012-12-09 13:21:29,319::task::978::TaskManager.Task::(_decref) Task=`481107bd-afb3-4cc2-860a-703522974f53`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:29,420::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:29,463::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174528 s, 58.7 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:21:30,685::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:30,686::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:30,786::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:30,787::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:30,787::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:30,787::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:31,470::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:31,519::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162777 s, 62.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:33,526::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:33,566::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162335 s, 63.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:21:35,158::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:35,165::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000390897 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:35,573::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:35,616::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171381 s, 59.7 MB/s\n'; <rc> = 0
>Thread-3439::DEBUG::2012-12-09 13:21:36,128::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3439::DEBUG::2012-12-09 13:21:36,129::task::588::TaskManager.Task::(_updateState) Task=`983ac149-4fcb-4fce-80fe-13ff2c7686eb`::moving from state init -> state preparing
>Thread-3439::INFO::2012-12-09 13:21:36,129::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3439::INFO::2012-12-09 13:21:36,129::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3439::DEBUG::2012-12-09 13:21:36,130::task::1172::TaskManager.Task::(prepare) Task=`983ac149-4fcb-4fce-80fe-13ff2c7686eb`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3439::DEBUG::2012-12-09 13:21:36,130::task::588::TaskManager.Task::(_updateState) Task=`983ac149-4fcb-4fce-80fe-13ff2c7686eb`::moving from state preparing -> state finished
>Thread-3439::DEBUG::2012-12-09 13:21:36,131::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3439::DEBUG::2012-12-09 13:21:36,131::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3439::DEBUG::2012-12-09 13:21:36,131::task::978::TaskManager.Task::(_decref) Task=`983ac149-4fcb-4fce-80fe-13ff2c7686eb`::ref 0 aborting False
>Thread-3440::DEBUG::2012-12-09 13:21:36,155::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3440::DEBUG::2012-12-09 13:21:36,156::task::588::TaskManager.Task::(_updateState) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::moving from state init -> state preparing
>Thread-3440::INFO::2012-12-09 13:21:36,157::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3440::DEBUG::2012-12-09 13:21:36,157::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7cf51f2-e935-4d7f-a6e0-3116ec6be7f9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3440::DEBUG::2012-12-09 13:21:36,157::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3440::DEBUG::2012-12-09 13:21:36,158::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3440::DEBUG::2012-12-09 13:21:36,158::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`d7cf51f2-e935-4d7f-a6e0-3116ec6be7f9`::Granted request
>Thread-3440::DEBUG::2012-12-09 13:21:36,159::task::817::TaskManager.Task::(resourceAcquired) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3440::DEBUG::2012-12-09 13:21:36,159::task::978::TaskManager.Task::(_decref) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::ref 1 aborting False
>Thread-3440::DEBUG::2012-12-09 13:21:36,161::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3440::DEBUG::2012-12-09 13:21:36,162::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3440::DEBUG::2012-12-09 13:21:36,258::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3440::WARNING::2012-12-09 13:21:36,259::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3440::DEBUG::2012-12-09 13:21:36,259::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3440::WARNING::2012-12-09 13:21:36,259::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3440::INFO::2012-12-09 13:21:36,260::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3440::DEBUG::2012-12-09 13:21:36,260::task::1172::TaskManager.Task::(prepare) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3440::DEBUG::2012-12-09 13:21:36,260::task::588::TaskManager.Task::(_updateState) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::moving from state preparing -> state finished
>Thread-3440::DEBUG::2012-12-09 13:21:36,260::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3440::DEBUG::2012-12-09 13:21:36,261::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3440::DEBUG::2012-12-09 13:21:36,261::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3440::DEBUG::2012-12-09 13:21:36,261::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3440::DEBUG::2012-12-09 13:21:36,261::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3440::DEBUG::2012-12-09 13:21:36,261::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3440::DEBUG::2012-12-09 13:21:36,262::task::978::TaskManager.Task::(_decref) Task=`f856b1f3-d2c3-439e-a7ba-68802a8396f3`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:37,623::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:37,675::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276045 s, 37.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:39,682::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:39,720::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0161412 s, 63.4 MB/s\n'; <rc> = 0
>Thread-3442::DEBUG::2012-12-09 13:21:40,085::task::588::TaskManager.Task::(_updateState) Task=`90432d56-f3c3-413d-a7e2-38eaede6076b`::moving from state init -> state preparing
>Thread-3442::INFO::2012-12-09 13:21:40,086::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3442::INFO::2012-12-09 13:21:40,087::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00759100914001', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-3442::DEBUG::2012-12-09 13:21:40,087::task::1172::TaskManager.Task::(prepare) Task=`90432d56-f3c3-413d-a7e2-38eaede6076b`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00759100914001', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.3', 'code': 358, 'valid': False}}
>Thread-3442::DEBUG::2012-12-09 13:21:40,088::task::588::TaskManager.Task::(_updateState) Task=`90432d56-f3c3-413d-a7e2-38eaede6076b`::moving from state preparing -> state finished
>Thread-3442::DEBUG::2012-12-09 13:21:40,088::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3442::DEBUG::2012-12-09 13:21:40,088::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3442::DEBUG::2012-12-09 13:21:40,089::task::978::TaskManager.Task::(_decref) Task=`90432d56-f3c3-413d-a7e2-38eaede6076b`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:21:40,788::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:40,789::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:40,886::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:40,887::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:40,887::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:40,887::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:41,727::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:41,767::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165617 s, 61.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:43,774::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:43,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192524 s, 53.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:21:45,168::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:45,183::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000773553 s, 5.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:45,824::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:45,864::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164323 s, 62.3 MB/s\n'; <rc> = 0
>Thread-3447::DEBUG::2012-12-09 13:21:46,798::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3447::DEBUG::2012-12-09 13:21:46,798::task::588::TaskManager.Task::(_updateState) Task=`ffdff803-4911-4687-9fd8-0e30bc6c1d48`::moving from state init -> state preparing
>Thread-3447::INFO::2012-12-09 13:21:46,798::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3447::INFO::2012-12-09 13:21:46,799::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3447::DEBUG::2012-12-09 13:21:46,799::task::1172::TaskManager.Task::(prepare) Task=`ffdff803-4911-4687-9fd8-0e30bc6c1d48`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3447::DEBUG::2012-12-09 13:21:46,799::task::588::TaskManager.Task::(_updateState) Task=`ffdff803-4911-4687-9fd8-0e30bc6c1d48`::moving from state preparing -> state finished
>Thread-3447::DEBUG::2012-12-09 13:21:46,799::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3447::DEBUG::2012-12-09 13:21:46,799::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3447::DEBUG::2012-12-09 13:21:46,800::task::978::TaskManager.Task::(_decref) Task=`ffdff803-4911-4687-9fd8-0e30bc6c1d48`::ref 0 aborting False
>Thread-3448::DEBUG::2012-12-09 13:21:46,814::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3448::DEBUG::2012-12-09 13:21:46,814::task::588::TaskManager.Task::(_updateState) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::moving from state init -> state preparing
>Thread-3448::INFO::2012-12-09 13:21:46,814::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3448::DEBUG::2012-12-09 13:21:46,815::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b71bea26-9d43-45fa-a6ab-bc6d180aa89e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3448::DEBUG::2012-12-09 13:21:46,815::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3448::DEBUG::2012-12-09 13:21:46,815::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3448::DEBUG::2012-12-09 13:21:46,815::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b71bea26-9d43-45fa-a6ab-bc6d180aa89e`::Granted request
>Thread-3448::DEBUG::2012-12-09 13:21:46,816::task::817::TaskManager.Task::(resourceAcquired) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3448::DEBUG::2012-12-09 13:21:46,816::task::978::TaskManager.Task::(_decref) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::ref 1 aborting False
>Thread-3448::DEBUG::2012-12-09 13:21:46,816::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3448::DEBUG::2012-12-09 13:21:46,817::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3448::DEBUG::2012-12-09 13:21:46,917::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3448::WARNING::2012-12-09 13:21:46,918::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3448::DEBUG::2012-12-09 13:21:46,918::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3448::WARNING::2012-12-09 13:21:46,919::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3448::INFO::2012-12-09 13:21:46,919::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3448::DEBUG::2012-12-09 13:21:46,919::task::1172::TaskManager.Task::(prepare) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3448::DEBUG::2012-12-09 13:21:46,920::task::588::TaskManager.Task::(_updateState) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::moving from state preparing -> state finished
>Thread-3448::DEBUG::2012-12-09 13:21:46,920::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3448::DEBUG::2012-12-09 13:21:46,920::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3448::DEBUG::2012-12-09 13:21:46,920::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3448::DEBUG::2012-12-09 13:21:46,921::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3448::DEBUG::2012-12-09 13:21:46,921::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3448::DEBUG::2012-12-09 13:21:46,921::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3448::DEBUG::2012-12-09 13:21:46,921::task::978::TaskManager.Task::(_decref) Task=`00b95105-8f6a-4470-831b-baf35f31adfc`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:21:47,870::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:47,911::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173834 s, 58.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:49,918::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:49,969::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265064 s, 38.6 MB/s\n'; <rc> = 0
>Thread-3450::DEBUG::2012-12-09 13:21:50,596::task::588::TaskManager.Task::(_updateState) Task=`48d00301-1b78-4cf3-b7ce-b2a9fa277e4e`::moving from state init -> state preparing
>Thread-3450::INFO::2012-12-09 13:21:50,597::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3450::INFO::2012-12-09 13:21:50,597::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154550075531', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-3450::DEBUG::2012-12-09 13:21:50,597::task::1172::TaskManager.Task::(prepare) Task=`48d00301-1b78-4cf3-b7ce-b2a9fa277e4e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0154550075531', 'lastCheck': '5.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '9.7', 'code': 358, 'valid': False}}
>Thread-3450::DEBUG::2012-12-09 13:21:50,598::task::588::TaskManager.Task::(_updateState) Task=`48d00301-1b78-4cf3-b7ce-b2a9fa277e4e`::moving from state preparing -> state finished
>Thread-3450::DEBUG::2012-12-09 13:21:50,598::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3450::DEBUG::2012-12-09 13:21:50,598::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3450::DEBUG::2012-12-09 13:21:50,598::task::978::TaskManager.Task::(_decref) Task=`48d00301-1b78-4cf3-b7ce-b2a9fa277e4e`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:21:50,888::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:21:50,889::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:21:50,984::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:21:50,985::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:21:50,985::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:21:50,985::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:21:51,976::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:52,014::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015656 s, 65.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:54,021::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:54,080::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0289833 s, 35.3 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:21:55,186::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:21:55,197::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000755691 s, 5.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:56,087::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:56,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0222691 s, 46.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:21:58,143::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:21:58,199::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0287741 s, 35.6 MB/s\n'; <rc> = 0
>Thread-3454::DEBUG::2012-12-09 13:21:58,427::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3454::DEBUG::2012-12-09 13:21:58,428::task::588::TaskManager.Task::(_updateState) Task=`3bac7ed1-2489-40d5-808f-466e7b3d8bc4`::moving from state init -> state preparing
>Thread-3454::INFO::2012-12-09 13:21:58,428::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3454::INFO::2012-12-09 13:21:58,429::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3454::DEBUG::2012-12-09 13:21:58,429::task::1172::TaskManager.Task::(prepare) Task=`3bac7ed1-2489-40d5-808f-466e7b3d8bc4`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3454::DEBUG::2012-12-09 13:21:58,429::task::588::TaskManager.Task::(_updateState) Task=`3bac7ed1-2489-40d5-808f-466e7b3d8bc4`::moving from state preparing -> state finished
>Thread-3454::DEBUG::2012-12-09 13:21:58,429::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3454::DEBUG::2012-12-09 13:21:58,429::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3454::DEBUG::2012-12-09 13:21:58,430::task::978::TaskManager.Task::(_decref) Task=`3bac7ed1-2489-40d5-808f-466e7b3d8bc4`::ref 0 aborting False
>Thread-3455::DEBUG::2012-12-09 13:21:58,442::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3455::DEBUG::2012-12-09 13:21:58,443::task::588::TaskManager.Task::(_updateState) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::moving from state init -> state preparing
>Thread-3455::INFO::2012-12-09 13:21:58,443::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3455::DEBUG::2012-12-09 13:21:58,444::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`38b398d9-3bf3-4454-8060-87bc33ca8738`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3455::DEBUG::2012-12-09 13:21:58,444::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3455::DEBUG::2012-12-09 13:21:58,445::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3455::DEBUG::2012-12-09 13:21:58,445::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`38b398d9-3bf3-4454-8060-87bc33ca8738`::Granted request
>Thread-3455::DEBUG::2012-12-09 13:21:58,446::task::817::TaskManager.Task::(resourceAcquired) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3455::DEBUG::2012-12-09 13:21:58,446::task::978::TaskManager.Task::(_decref) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::ref 1 aborting False
>Thread-3455::DEBUG::2012-12-09 13:21:58,448::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3455::DEBUG::2012-12-09 13:21:58,449::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3455::DEBUG::2012-12-09 13:21:58,552::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3455::WARNING::2012-12-09 13:21:58,553::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3455::DEBUG::2012-12-09 13:21:58,553::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3455::WARNING::2012-12-09 13:21:58,554::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3455::INFO::2012-12-09 13:21:58,554::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3455::DEBUG::2012-12-09 13:21:58,554::task::1172::TaskManager.Task::(prepare) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3455::DEBUG::2012-12-09 13:21:58,554::task::588::TaskManager.Task::(_updateState) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::moving from state preparing -> state finished
>Thread-3455::DEBUG::2012-12-09 13:21:58,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3455::DEBUG::2012-12-09 13:21:58,555::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3455::DEBUG::2012-12-09 13:21:58,555::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3455::DEBUG::2012-12-09 13:21:58,555::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3455::DEBUG::2012-12-09 13:21:58,555::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3455::DEBUG::2012-12-09 13:21:58,556::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3455::DEBUG::2012-12-09 13:21:58,556::task::978::TaskManager.Task::(_decref) Task=`39b3eaa1-c5e1-43e2-8322-aa60b74e8055`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:00,206::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:00,247::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169975 s, 60.2 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:22:00,986::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:00,988::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:01,101::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:01,103::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:01,103::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:22:01,104::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:22:02,254::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:02,299::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203646 s, 50.3 MB/s\n'; <rc> = 0
>Thread-3458::DEBUG::2012-12-09 13:22:02,742::task::588::TaskManager.Task::(_updateState) Task=`4f65653b-aec4-401f-bba3-a87894dc44a6`::moving from state init -> state preparing
>Thread-3458::INFO::2012-12-09 13:22:02,742::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3458::INFO::2012-12-09 13:22:02,743::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122818946838', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-3458::DEBUG::2012-12-09 13:22:02,743::task::1172::TaskManager.Task::(prepare) Task=`4f65653b-aec4-401f-bba3-a87894dc44a6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0122818946838', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False}}
>Thread-3458::DEBUG::2012-12-09 13:22:02,743::task::588::TaskManager.Task::(_updateState) Task=`4f65653b-aec4-401f-bba3-a87894dc44a6`::moving from state preparing -> state finished
>Thread-3458::DEBUG::2012-12-09 13:22:02,743::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3458::DEBUG::2012-12-09 13:22:02,743::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3458::DEBUG::2012-12-09 13:22:02,744::task::978::TaskManager.Task::(_decref) Task=`4f65653b-aec4-401f-bba3-a87894dc44a6`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:04,306::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:04,356::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0254923 s, 40.2 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:22:05,203::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:05,218::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000865755 s, 4.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:06,363::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:06,415::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.026896 s, 38.1 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:08,421::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:08,462::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168946 s, 60.6 MB/s\n'; <rc> = 0
>Thread-3462::DEBUG::2012-12-09 13:22:09,177::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3462::DEBUG::2012-12-09 13:22:09,177::task::588::TaskManager.Task::(_updateState) Task=`45c629e6-dfde-4e84-911f-0dae29f99a19`::moving from state init -> state preparing
>Thread-3462::INFO::2012-12-09 13:22:09,178::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3462::INFO::2012-12-09 13:22:09,178::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3462::DEBUG::2012-12-09 13:22:09,178::task::1172::TaskManager.Task::(prepare) Task=`45c629e6-dfde-4e84-911f-0dae29f99a19`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3462::DEBUG::2012-12-09 13:22:09,178::task::588::TaskManager.Task::(_updateState) Task=`45c629e6-dfde-4e84-911f-0dae29f99a19`::moving from state preparing -> state finished
>Thread-3462::DEBUG::2012-12-09 13:22:09,178::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3462::DEBUG::2012-12-09 13:22:09,179::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3462::DEBUG::2012-12-09 13:22:09,179::task::978::TaskManager.Task::(_decref) Task=`45c629e6-dfde-4e84-911f-0dae29f99a19`::ref 0 aborting False
>Thread-3463::DEBUG::2012-12-09 13:22:09,198::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3463::DEBUG::2012-12-09 13:22:09,198::task::588::TaskManager.Task::(_updateState) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::moving from state init -> state preparing
>Thread-3463::INFO::2012-12-09 13:22:09,199::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3463::DEBUG::2012-12-09 13:22:09,200::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de98fae7-1d2e-439e-a6f8-176fa54fd538`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3463::DEBUG::2012-12-09 13:22:09,200::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3463::DEBUG::2012-12-09 13:22:09,201::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3463::DEBUG::2012-12-09 13:22:09,201::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`de98fae7-1d2e-439e-a6f8-176fa54fd538`::Granted request
>Thread-3463::DEBUG::2012-12-09 13:22:09,202::task::817::TaskManager.Task::(resourceAcquired) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3463::DEBUG::2012-12-09 13:22:09,202::task::978::TaskManager.Task::(_decref) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::ref 1 aborting False
>Thread-3463::DEBUG::2012-12-09 13:22:09,203::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3463::DEBUG::2012-12-09 13:22:09,204::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3463::DEBUG::2012-12-09 13:22:09,299::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3463::WARNING::2012-12-09 13:22:09,301::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3463::DEBUG::2012-12-09 13:22:09,302::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3463::WARNING::2012-12-09 13:22:09,303::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3463::INFO::2012-12-09 13:22:09,303::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3463::DEBUG::2012-12-09 13:22:09,304::task::1172::TaskManager.Task::(prepare) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3463::DEBUG::2012-12-09 13:22:09,304::task::588::TaskManager.Task::(_updateState) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::moving from state preparing -> state finished
>Thread-3463::DEBUG::2012-12-09 13:22:09,305::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3463::DEBUG::2012-12-09 13:22:09,305::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3463::DEBUG::2012-12-09 13:22:09,306::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3463::DEBUG::2012-12-09 13:22:09,307::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3463::DEBUG::2012-12-09 13:22:09,307::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3463::DEBUG::2012-12-09 13:22:09,307::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3463::DEBUG::2012-12-09 13:22:09,308::task::978::TaskManager.Task::(_decref) Task=`f27c5d5a-330d-4807-9cf8-65872f97fdc4`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:10,469::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:10,517::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018661 s, 54.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:22:11,105::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:11,106::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:11,218::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:11,219::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:11,219::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:22:11,220::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:22:12,524::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:12,567::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178348 s, 57.4 MB/s\n'; <rc> = 0
>Thread-3466::DEBUG::2012-12-09 13:22:13,973::task::588::TaskManager.Task::(_updateState) Task=`f15f2c49-b381-49db-a919-76a6c06534cb`::moving from state init -> state preparing
>Thread-3466::INFO::2012-12-09 13:22:13,974::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3466::INFO::2012-12-09 13:22:13,974::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163588523865', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-3466::DEBUG::2012-12-09 13:22:13,974::task::1172::TaskManager.Task::(prepare) Task=`f15f2c49-b381-49db-a919-76a6c06534cb`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163588523865', 'lastCheck': '8.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '2.8', 'code': 358, 'valid': False}}
>Thread-3466::DEBUG::2012-12-09 13:22:13,974::task::588::TaskManager.Task::(_updateState) Task=`f15f2c49-b381-49db-a919-76a6c06534cb`::moving from state preparing -> state finished
>Thread-3466::DEBUG::2012-12-09 13:22:13,975::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3466::DEBUG::2012-12-09 13:22:13,975::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3466::DEBUG::2012-12-09 13:22:13,975::task::978::TaskManager.Task::(_decref) Task=`f15f2c49-b381-49db-a919-76a6c06534cb`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:14,574::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:14,616::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173305 s, 59.1 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:22:15,224::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:15,232::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000582043 s, 7.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:16,622::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:16,669::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236541 s, 43.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:18,676::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:18,715::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164052 s, 62.4 MB/s\n'; <rc> = 0
>Thread-3470::DEBUG::2012-12-09 13:22:19,958::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3470::DEBUG::2012-12-09 13:22:19,959::task::588::TaskManager.Task::(_updateState) Task=`b25ded85-6cfe-4e0d-a133-4ca50fed7e09`::moving from state init -> state preparing
>Thread-3470::INFO::2012-12-09 13:22:19,960::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3470::INFO::2012-12-09 13:22:19,961::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3470::DEBUG::2012-12-09 13:22:19,961::task::1172::TaskManager.Task::(prepare) Task=`b25ded85-6cfe-4e0d-a133-4ca50fed7e09`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3470::DEBUG::2012-12-09 13:22:19,961::task::588::TaskManager.Task::(_updateState) Task=`b25ded85-6cfe-4e0d-a133-4ca50fed7e09`::moving from state preparing -> state finished
>Thread-3470::DEBUG::2012-12-09 13:22:19,962::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3470::DEBUG::2012-12-09 13:22:19,962::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3470::DEBUG::2012-12-09 13:22:19,963::task::978::TaskManager.Task::(_decref) Task=`b25ded85-6cfe-4e0d-a133-4ca50fed7e09`::ref 0 aborting False
>Thread-3471::DEBUG::2012-12-09 13:22:19,980::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3471::DEBUG::2012-12-09 13:22:19,980::task::588::TaskManager.Task::(_updateState) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::moving from state init -> state preparing
>Thread-3471::INFO::2012-12-09 13:22:19,980::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3471::DEBUG::2012-12-09 13:22:19,981::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`153deeaf-5657-43b2-8642-0011fc69e921`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3471::DEBUG::2012-12-09 13:22:19,981::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3471::DEBUG::2012-12-09 13:22:19,981::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3471::DEBUG::2012-12-09 13:22:19,981::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`153deeaf-5657-43b2-8642-0011fc69e921`::Granted request
>Thread-3471::DEBUG::2012-12-09 13:22:19,982::task::817::TaskManager.Task::(resourceAcquired) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3471::DEBUG::2012-12-09 13:22:19,982::task::978::TaskManager.Task::(_decref) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::ref 1 aborting False
>Thread-3471::DEBUG::2012-12-09 13:22:19,983::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3471::DEBUG::2012-12-09 13:22:19,983::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3471::DEBUG::2012-12-09 13:22:20,097::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3471::WARNING::2012-12-09 13:22:20,100::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3471::DEBUG::2012-12-09 13:22:20,100::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3471::WARNING::2012-12-09 13:22:20,101::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3471::INFO::2012-12-09 13:22:20,102::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3471::DEBUG::2012-12-09 13:22:20,102::task::1172::TaskManager.Task::(prepare) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3471::DEBUG::2012-12-09 13:22:20,103::task::588::TaskManager.Task::(_updateState) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::moving from state preparing -> state finished
>Thread-3471::DEBUG::2012-12-09 13:22:20,103::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3471::DEBUG::2012-12-09 13:22:20,103::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3471::DEBUG::2012-12-09 13:22:20,104::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3471::DEBUG::2012-12-09 13:22:20,104::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3471::DEBUG::2012-12-09 13:22:20,104::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3471::DEBUG::2012-12-09 13:22:20,104::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3471::DEBUG::2012-12-09 13:22:20,105::task::978::TaskManager.Task::(_decref) Task=`e4b04168-8987-48c4-a966-68864b3e23df`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:20,722::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:20,764::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183247 s, 55.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:22:21,221::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:21,222::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:21,334::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:21,335::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:21,335::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:22:21,335::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:22:22,771::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:22,812::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178596 s, 57.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:24,819::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:24,859::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170556 s, 60.0 MB/s\n'; <rc> = 0
>Thread-3474::DEBUG::2012-12-09 13:22:25,185::task::588::TaskManager.Task::(_updateState) Task=`f1a477cf-7f70-4e5f-a68d-a24c2d803298`::moving from state init -> state preparing
>Thread-3474::INFO::2012-12-09 13:22:25,185::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3474::INFO::2012-12-09 13:22:25,185::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00852799415588', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-3474::DEBUG::2012-12-09 13:22:25,186::task::1172::TaskManager.Task::(prepare) Task=`f1a477cf-7f70-4e5f-a68d-a24c2d803298`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00852799415588', 'lastCheck': '10.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '3.8', 'code': 358, 'valid': False}}
>Thread-3474::DEBUG::2012-12-09 13:22:25,186::task::588::TaskManager.Task::(_updateState) Task=`f1a477cf-7f70-4e5f-a68d-a24c2d803298`::moving from state preparing -> state finished
>Thread-3474::DEBUG::2012-12-09 13:22:25,186::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3474::DEBUG::2012-12-09 13:22:25,186::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3474::DEBUG::2012-12-09 13:22:25,186::task::978::TaskManager.Task::(_decref) Task=`f1a477cf-7f70-4e5f-a68d-a24c2d803298`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:22:25,235::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:25,243::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000454954 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:26,865::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:26,912::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021124 s, 48.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:28,919::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:28,959::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166709 s, 61.4 MB/s\n'; <rc> = 0
>Thread-3478::DEBUG::2012-12-09 13:22:30,858::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3478::DEBUG::2012-12-09 13:22:30,859::task::588::TaskManager.Task::(_updateState) Task=`fe1a4dd7-5a97-4b21-8619-d9c836a47735`::moving from state init -> state preparing
>Thread-3478::INFO::2012-12-09 13:22:30,859::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3478::INFO::2012-12-09 13:22:30,860::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3478::DEBUG::2012-12-09 13:22:30,860::task::1172::TaskManager.Task::(prepare) Task=`fe1a4dd7-5a97-4b21-8619-d9c836a47735`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3478::DEBUG::2012-12-09 13:22:30,861::task::588::TaskManager.Task::(_updateState) Task=`fe1a4dd7-5a97-4b21-8619-d9c836a47735`::moving from state preparing -> state finished
>Thread-3478::DEBUG::2012-12-09 13:22:30,861::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3478::DEBUG::2012-12-09 13:22:30,862::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3478::DEBUG::2012-12-09 13:22:30,862::task::978::TaskManager.Task::(_decref) Task=`fe1a4dd7-5a97-4b21-8619-d9c836a47735`::ref 0 aborting False
>Thread-3479::DEBUG::2012-12-09 13:22:30,885::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3479::DEBUG::2012-12-09 13:22:30,886::task::588::TaskManager.Task::(_updateState) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::moving from state init -> state preparing
>Thread-3479::INFO::2012-12-09 13:22:30,886::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3479::DEBUG::2012-12-09 13:22:30,887::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3b8ffb53-96ba-4b7a-ac15-945bbec76c1d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3479::DEBUG::2012-12-09 13:22:30,887::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3479::DEBUG::2012-12-09 13:22:30,888::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3479::DEBUG::2012-12-09 13:22:30,888::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`3b8ffb53-96ba-4b7a-ac15-945bbec76c1d`::Granted request
>Thread-3479::DEBUG::2012-12-09 13:22:30,889::task::817::TaskManager.Task::(resourceAcquired) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3479::DEBUG::2012-12-09 13:22:30,890::task::978::TaskManager.Task::(_decref) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::ref 1 aborting False
>Thread-3479::DEBUG::2012-12-09 13:22:30,891::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3479::DEBUG::2012-12-09 13:22:30,892::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:30,966::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3479::DEBUG::2012-12-09 13:22:31,007::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3479::WARNING::2012-12-09 13:22:31,011::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Dummy-321::DEBUG::2012-12-09 13:22:31,012::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163534 s, 62.6 MB/s\n'; <rc> = 0
>Thread-3479::DEBUG::2012-12-09 13:22:31,012::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3479::WARNING::2012-12-09 13:22:31,022::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3479::INFO::2012-12-09 13:22:31,028::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3479::DEBUG::2012-12-09 13:22:31,031::task::1172::TaskManager.Task::(prepare) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3479::DEBUG::2012-12-09 13:22:31,032::task::588::TaskManager.Task::(_updateState) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::moving from state preparing -> state finished
>Thread-3479::DEBUG::2012-12-09 13:22:31,032::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3479::DEBUG::2012-12-09 13:22:31,032::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3479::DEBUG::2012-12-09 13:22:31,032::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3479::DEBUG::2012-12-09 13:22:31,033::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3479::DEBUG::2012-12-09 13:22:31,033::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3479::DEBUG::2012-12-09 13:22:31,033::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3479::DEBUG::2012-12-09 13:22:31,033::task::978::TaskManager.Task::(_decref) Task=`6b86e3c4-587b-4905-8952-58a8454664fa`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:22:31,336::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:31,337::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:31,438::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:31,440::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:31,440::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:22:31,440::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:22:33,034::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:33,082::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218594 s, 46.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:35,090::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:35,135::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216197 s, 47.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:22:35,246::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:35,258::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00103916 s, 3.9 MB/s\n'; <rc> = 0
>Thread-3482::DEBUG::2012-12-09 13:22:35,608::task::588::TaskManager.Task::(_updateState) Task=`123fa6ad-ec31-4b6b-af05-f08ce093b5be`::moving from state init -> state preparing
>Thread-3482::INFO::2012-12-09 13:22:35,608::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3482::INFO::2012-12-09 13:22:35,609::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135610103607', 'lastCheck': '0.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-3482::DEBUG::2012-12-09 13:22:35,609::task::1172::TaskManager.Task::(prepare) Task=`123fa6ad-ec31-4b6b-af05-f08ce093b5be`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0135610103607', 'lastCheck': '0.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.2', 'code': 358, 'valid': False}}
>Thread-3482::DEBUG::2012-12-09 13:22:35,609::task::588::TaskManager.Task::(_updateState) Task=`123fa6ad-ec31-4b6b-af05-f08ce093b5be`::moving from state preparing -> state finished
>Thread-3482::DEBUG::2012-12-09 13:22:35,609::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3482::DEBUG::2012-12-09 13:22:35,609::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3482::DEBUG::2012-12-09 13:22:35,610::task::978::TaskManager.Task::(_decref) Task=`123fa6ad-ec31-4b6b-af05-f08ce093b5be`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:37,141::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:37,198::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0251305 s, 40.7 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:39,205::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:39,253::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220085 s, 46.5 MB/s\n'; <rc> = 0
>Thread-3486::DEBUG::2012-12-09 13:22:41,266::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Dummy-321::DEBUG::2012-12-09 13:22:41,266::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-3486::DEBUG::2012-12-09 13:22:41,268::task::588::TaskManager.Task::(_updateState) Task=`cc001c19-f646-4d0a-ae1f-c7113a8cd08f`::moving from state init -> state preparing
>Thread-3486::INFO::2012-12-09 13:22:41,276::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3486::INFO::2012-12-09 13:22:41,276::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3486::DEBUG::2012-12-09 13:22:41,276::task::1172::TaskManager.Task::(prepare) Task=`cc001c19-f646-4d0a-ae1f-c7113a8cd08f`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3486::DEBUG::2012-12-09 13:22:41,277::task::588::TaskManager.Task::(_updateState) Task=`cc001c19-f646-4d0a-ae1f-c7113a8cd08f`::moving from state preparing -> state finished
>Thread-3486::DEBUG::2012-12-09 13:22:41,277::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3486::DEBUG::2012-12-09 13:22:41,277::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3486::DEBUG::2012-12-09 13:22:41,277::task::978::TaskManager.Task::(_decref) Task=`cc001c19-f646-4d0a-ae1f-c7113a8cd08f`::ref 0 aborting False
>Thread-3487::DEBUG::2012-12-09 13:22:41,295::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3487::DEBUG::2012-12-09 13:22:41,295::task::588::TaskManager.Task::(_updateState) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::moving from state init -> state preparing
>Thread-3487::INFO::2012-12-09 13:22:41,296::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3487::DEBUG::2012-12-09 13:22:41,297::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ab8d3aa4-eba3-4ec2-97fd-dde7c2c419bb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3487::DEBUG::2012-12-09 13:22:41,297::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3487::DEBUG::2012-12-09 13:22:41,298::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3487::DEBUG::2012-12-09 13:22:41,300::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ab8d3aa4-eba3-4ec2-97fd-dde7c2c419bb`::Granted request
>Thread-3487::DEBUG::2012-12-09 13:22:41,301::task::817::TaskManager.Task::(resourceAcquired) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3487::DEBUG::2012-12-09 13:22:41,302::task::978::TaskManager.Task::(_decref) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::ref 1 aborting False
>Thread-3487::DEBUG::2012-12-09 13:22:41,304::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3487::DEBUG::2012-12-09 13:22:41,306::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:41,335::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376816 s, 27.2 MB/s\n'; <rc> = 0
>Thread-3487::DEBUG::2012-12-09 13:22:41,411::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3487::WARNING::2012-12-09 13:22:41,412::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3487::DEBUG::2012-12-09 13:22:41,413::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3487::WARNING::2012-12-09 13:22:41,414::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3487::INFO::2012-12-09 13:22:41,414::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3487::DEBUG::2012-12-09 13:22:41,414::task::1172::TaskManager.Task::(prepare) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3487::DEBUG::2012-12-09 13:22:41,415::task::588::TaskManager.Task::(_updateState) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::moving from state preparing -> state finished
>Thread-3487::DEBUG::2012-12-09 13:22:41,415::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3487::DEBUG::2012-12-09 13:22:41,415::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3487::DEBUG::2012-12-09 13:22:41,416::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3487::DEBUG::2012-12-09 13:22:41,416::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3487::DEBUG::2012-12-09 13:22:41,416::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3487::DEBUG::2012-12-09 13:22:41,416::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3487::DEBUG::2012-12-09 13:22:41,416::task::978::TaskManager.Task::(_decref) Task=`c143c2be-0d51-4be7-abb8-7963953657fc`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:22:41,441::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:41,442::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:41,553::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:41,555::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:41,555::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:22:41,555::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:22:43,341::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:43,394::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250631 s, 40.9 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:22:45,264::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:45,274::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000543738 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:45,400::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:45,441::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168469 s, 60.8 MB/s\n'; <rc> = 0
>Thread-3490::DEBUG::2012-12-09 13:22:45,817::task::588::TaskManager.Task::(_updateState) Task=`520b5478-3fe0-4748-909b-35d05271f78a`::moving from state init -> state preparing
>Thread-3490::INFO::2012-12-09 13:22:45,818::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3490::INFO::2012-12-09 13:22:45,818::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0100650787354', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-3490::DEBUG::2012-12-09 13:22:45,819::task::1172::TaskManager.Task::(prepare) Task=`520b5478-3fe0-4748-909b-35d05271f78a`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0100650787354', 'lastCheck': '0.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.3', 'code': 358, 'valid': False}}
>Thread-3490::DEBUG::2012-12-09 13:22:45,819::task::588::TaskManager.Task::(_updateState) Task=`520b5478-3fe0-4748-909b-35d05271f78a`::moving from state preparing -> state finished
>Thread-3490::DEBUG::2012-12-09 13:22:45,820::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3490::DEBUG::2012-12-09 13:22:45,820::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3490::DEBUG::2012-12-09 13:22:45,820::task::978::TaskManager.Task::(_decref) Task=`520b5478-3fe0-4748-909b-35d05271f78a`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:47,448::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:47,498::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0247113 s, 41.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:49,505::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:49,544::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163161 s, 62.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:51,550::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:22:51,556::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:22:51,557::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3494::DEBUG::2012-12-09 13:22:51,565::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3494::DEBUG::2012-12-09 13:22:51,566::task::588::TaskManager.Task::(_updateState) Task=`2b78283b-5ceb-4c5f-b08e-0142119ca91c`::moving from state init -> state preparing
>Thread-3494::INFO::2012-12-09 13:22:51,581::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3494::INFO::2012-12-09 13:22:51,582::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3494::DEBUG::2012-12-09 13:22:51,582::task::1172::TaskManager.Task::(prepare) Task=`2b78283b-5ceb-4c5f-b08e-0142119ca91c`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3494::DEBUG::2012-12-09 13:22:51,582::task::588::TaskManager.Task::(_updateState) Task=`2b78283b-5ceb-4c5f-b08e-0142119ca91c`::moving from state preparing -> state finished
>Thread-3494::DEBUG::2012-12-09 13:22:51,582::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3494::DEBUG::2012-12-09 13:22:51,582::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3494::DEBUG::2012-12-09 13:22:51,583::task::978::TaskManager.Task::(_decref) Task=`2b78283b-5ceb-4c5f-b08e-0142119ca91c`::ref 0 aborting False
>Thread-3495::DEBUG::2012-12-09 13:22:51,602::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3495::DEBUG::2012-12-09 13:22:51,604::task::588::TaskManager.Task::(_updateState) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::moving from state init -> state preparing
>Thread-3495::INFO::2012-12-09 13:22:51,605::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Dummy-321::DEBUG::2012-12-09 13:22:51,611::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265274 s, 38.6 MB/s\n'; <rc> = 0
>Thread-3495::DEBUG::2012-12-09 13:22:51,613::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff3a9951-7d78-4b9d-9c14-a3c27de80cd9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3495::DEBUG::2012-12-09 13:22:51,615::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3495::DEBUG::2012-12-09 13:22:51,620::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3495::DEBUG::2012-12-09 13:22:51,626::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`ff3a9951-7d78-4b9d-9c14-a3c27de80cd9`::Granted request
>Thread-3495::DEBUG::2012-12-09 13:22:51,626::task::817::TaskManager.Task::(resourceAcquired) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3495::DEBUG::2012-12-09 13:22:51,627::task::978::TaskManager.Task::(_decref) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::ref 1 aborting False
>Thread-51::DEBUG::2012-12-09 13:22:51,672::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:22:51,673::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:22:51,673::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3495::DEBUG::2012-12-09 13:22:51,674::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3495::DEBUG::2012-12-09 13:22:51,674::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::ERROR::2012-12-09 13:22:51,679::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3495::DEBUG::2012-12-09 13:22:51,787::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3495::WARNING::2012-12-09 13:22:51,789::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3495::DEBUG::2012-12-09 13:22:51,790::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3495::WARNING::2012-12-09 13:22:51,791::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3495::INFO::2012-12-09 13:22:51,791::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3495::DEBUG::2012-12-09 13:22:51,792::task::1172::TaskManager.Task::(prepare) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3495::DEBUG::2012-12-09 13:22:51,792::task::588::TaskManager.Task::(_updateState) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::moving from state preparing -> state finished
>Thread-3495::DEBUG::2012-12-09 13:22:51,793::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3495::DEBUG::2012-12-09 13:22:51,793::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3495::DEBUG::2012-12-09 13:22:51,793::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3495::DEBUG::2012-12-09 13:22:51,794::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3495::DEBUG::2012-12-09 13:22:51,794::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3495::DEBUG::2012-12-09 13:22:51,794::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3495::DEBUG::2012-12-09 13:22:51,794::task::978::TaskManager.Task::(_decref) Task=`b2a31c61-47a6-4693-b968-3dd498258046`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:53,624::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:53,663::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155142 s, 66.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:22:55,278::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:22:55,289::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000397281 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:55,670::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:55,715::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181469 s, 56.4 MB/s\n'; <rc> = 0
>Thread-3498::DEBUG::2012-12-09 13:22:56,109::task::588::TaskManager.Task::(_updateState) Task=`9d4657aa-04fc-4a7e-9f30-85e34aae3fb7`::moving from state init -> state preparing
>Thread-3498::INFO::2012-12-09 13:22:56,110::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3498::INFO::2012-12-09 13:22:56,110::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118780136108', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-3498::DEBUG::2012-12-09 13:22:56,110::task::1172::TaskManager.Task::(prepare) Task=`9d4657aa-04fc-4a7e-9f30-85e34aae3fb7`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0118780136108', 'lastCheck': '0.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '4.4', 'code': 358, 'valid': False}}
>Thread-3498::DEBUG::2012-12-09 13:22:56,110::task::588::TaskManager.Task::(_updateState) Task=`9d4657aa-04fc-4a7e-9f30-85e34aae3fb7`::moving from state preparing -> state finished
>Thread-3498::DEBUG::2012-12-09 13:22:56,110::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3498::DEBUG::2012-12-09 13:22:56,111::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3498::DEBUG::2012-12-09 13:22:56,111::task::978::TaskManager.Task::(_decref) Task=`9d4657aa-04fc-4a7e-9f30-85e34aae3fb7`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:22:57,722::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:57,770::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228262 s, 44.9 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:22:59,777::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:22:59,829::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270539 s, 37.9 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:23:01,680::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:01,682::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:01,794::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:01,795::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:01,795::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:01,796::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:23:01,836::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:01,880::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202882 s, 50.5 MB/s\n'; <rc> = 0
>Thread-3502::DEBUG::2012-12-09 13:23:01,945::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3502::DEBUG::2012-12-09 13:23:01,946::task::588::TaskManager.Task::(_updateState) Task=`848961bd-cef0-4e31-a0ab-d06e25800275`::moving from state init -> state preparing
>Thread-3502::INFO::2012-12-09 13:23:01,946::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3502::INFO::2012-12-09 13:23:01,946::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3502::DEBUG::2012-12-09 13:23:01,946::task::1172::TaskManager.Task::(prepare) Task=`848961bd-cef0-4e31-a0ab-d06e25800275`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3502::DEBUG::2012-12-09 13:23:01,947::task::588::TaskManager.Task::(_updateState) Task=`848961bd-cef0-4e31-a0ab-d06e25800275`::moving from state preparing -> state finished
>Thread-3502::DEBUG::2012-12-09 13:23:01,947::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3502::DEBUG::2012-12-09 13:23:01,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3502::DEBUG::2012-12-09 13:23:01,947::task::978::TaskManager.Task::(_decref) Task=`848961bd-cef0-4e31-a0ab-d06e25800275`::ref 0 aborting False
>Thread-3503::DEBUG::2012-12-09 13:23:01,967::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3503::DEBUG::2012-12-09 13:23:01,967::task::588::TaskManager.Task::(_updateState) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::moving from state init -> state preparing
>Thread-3503::INFO::2012-12-09 13:23:01,968::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3503::DEBUG::2012-12-09 13:23:01,968::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b59e110b-dc21-49b9-bf13-9aaa07030357`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3503::DEBUG::2012-12-09 13:23:01,968::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3503::DEBUG::2012-12-09 13:23:01,969::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3503::DEBUG::2012-12-09 13:23:01,969::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`b59e110b-dc21-49b9-bf13-9aaa07030357`::Granted request
>Thread-3503::DEBUG::2012-12-09 13:23:01,969::task::817::TaskManager.Task::(resourceAcquired) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3503::DEBUG::2012-12-09 13:23:01,970::task::978::TaskManager.Task::(_decref) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::ref 1 aborting False
>Thread-3503::DEBUG::2012-12-09 13:23:01,971::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3503::DEBUG::2012-12-09 13:23:01,972::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3503::DEBUG::2012-12-09 13:23:02,082::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3503::WARNING::2012-12-09 13:23:02,084::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3503::DEBUG::2012-12-09 13:23:02,084::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3503::WARNING::2012-12-09 13:23:02,084::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3503::INFO::2012-12-09 13:23:02,085::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3503::DEBUG::2012-12-09 13:23:02,085::task::1172::TaskManager.Task::(prepare) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3503::DEBUG::2012-12-09 13:23:02,085::task::588::TaskManager.Task::(_updateState) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::moving from state preparing -> state finished
>Thread-3503::DEBUG::2012-12-09 13:23:02,085::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3503::DEBUG::2012-12-09 13:23:02,086::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3503::DEBUG::2012-12-09 13:23:02,086::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3503::DEBUG::2012-12-09 13:23:02,086::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3503::DEBUG::2012-12-09 13:23:02,086::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3503::DEBUG::2012-12-09 13:23:02,087::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3503::DEBUG::2012-12-09 13:23:02,087::task::978::TaskManager.Task::(_decref) Task=`30d9e34b-314c-44de-9437-15f6053e9a83`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:23:03,886::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:03,937::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0255935 s, 40.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:23:05,292::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:05,305::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000749067 s, 5.5 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:23:05,943::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:05,983::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164301 s, 62.3 MB/s\n'; <rc> = 0
>Thread-3506::DEBUG::2012-12-09 13:23:07,270::task::588::TaskManager.Task::(_updateState) Task=`296e4cef-f5ff-40d9-a828-ec193a2252bd`::moving from state init -> state preparing
>Thread-3506::INFO::2012-12-09 13:23:07,271::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3506::INFO::2012-12-09 13:23:07,271::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133190155029', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-3506::DEBUG::2012-12-09 13:23:07,271::task::1172::TaskManager.Task::(prepare) Task=`296e4cef-f5ff-40d9-a828-ec193a2252bd`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133190155029', 'lastCheck': '2.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.5', 'code': 358, 'valid': False}}
>Thread-3506::DEBUG::2012-12-09 13:23:07,271::task::588::TaskManager.Task::(_updateState) Task=`296e4cef-f5ff-40d9-a828-ec193a2252bd`::moving from state preparing -> state finished
>Thread-3506::DEBUG::2012-12-09 13:23:07,271::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3506::DEBUG::2012-12-09 13:23:07,272::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3506::DEBUG::2012-12-09 13:23:07,272::task::978::TaskManager.Task::(_decref) Task=`296e4cef-f5ff-40d9-a828-ec193a2252bd`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:23:07,990::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:08,038::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235906 s, 43.4 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:23:10,045::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:10,087::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166765 s, 61.4 MB/s\n'; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:23:11,796::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:11,797::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:11,901::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:11,903::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:11,903::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:11,904::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Dummy-321::DEBUG::2012-12-09 13:23:12,093::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:12,131::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166236 s, 61.6 MB/s\n'; <rc> = 0
>Thread-3510::DEBUG::2012-12-09 13:23:12,190::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3510::DEBUG::2012-12-09 13:23:12,191::task::588::TaskManager.Task::(_updateState) Task=`4ca26160-e578-42c0-a886-d49a2b4dfc36`::moving from state init -> state preparing
>Thread-3510::INFO::2012-12-09 13:23:12,191::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3510::INFO::2012-12-09 13:23:12,192::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3510::DEBUG::2012-12-09 13:23:12,192::task::1172::TaskManager.Task::(prepare) Task=`4ca26160-e578-42c0-a886-d49a2b4dfc36`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3510::DEBUG::2012-12-09 13:23:12,192::task::588::TaskManager.Task::(_updateState) Task=`4ca26160-e578-42c0-a886-d49a2b4dfc36`::moving from state preparing -> state finished
>Thread-3510::DEBUG::2012-12-09 13:23:12,192::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3510::DEBUG::2012-12-09 13:23:12,192::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3510::DEBUG::2012-12-09 13:23:12,193::task::978::TaskManager.Task::(_decref) Task=`4ca26160-e578-42c0-a886-d49a2b4dfc36`::ref 0 aborting False
>Thread-3511::DEBUG::2012-12-09 13:23:12,207::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3511::DEBUG::2012-12-09 13:23:12,208::task::588::TaskManager.Task::(_updateState) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::moving from state init -> state preparing
>Thread-3511::INFO::2012-12-09 13:23:12,208::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3511::DEBUG::2012-12-09 13:23:12,208::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`87a88e91-c43f-47c7-bc2d-4af98a17445f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3511::DEBUG::2012-12-09 13:23:12,209::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3511::DEBUG::2012-12-09 13:23:12,209::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3511::DEBUG::2012-12-09 13:23:12,209::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`87a88e91-c43f-47c7-bc2d-4af98a17445f`::Granted request
>Thread-3511::DEBUG::2012-12-09 13:23:12,209::task::817::TaskManager.Task::(resourceAcquired) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3511::DEBUG::2012-12-09 13:23:12,210::task::978::TaskManager.Task::(_decref) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::ref 1 aborting False
>Thread-3511::DEBUG::2012-12-09 13:23:12,210::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3511::DEBUG::2012-12-09 13:23:12,211::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3511::DEBUG::2012-12-09 13:23:12,320::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3511::WARNING::2012-12-09 13:23:12,322::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3511::DEBUG::2012-12-09 13:23:12,322::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3511::WARNING::2012-12-09 13:23:12,322::sp::1447::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1442, in getInfo
>    dom = sdCache.produce(item)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-3511::INFO::2012-12-09 13:23:12,323::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3511::DEBUG::2012-12-09 13:23:12,323::task::1172::TaskManager.Task::(prepare) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::finished: {'info': {'spm_id': 2, 'master_uuid': 'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'name': 'Default', 'version': '3', 'domains': u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'ISCSI', 'master_ver': 1, 'lver': 49}, 'dominfo': {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'status': u'Active', 'diskfree': '80262201344', 'alerts': [], 'disktotal': '106971529216'}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'status': u'Active', 'alerts': []}}}
>Thread-3511::DEBUG::2012-12-09 13:23:12,323::task::588::TaskManager.Task::(_updateState) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::moving from state preparing -> state finished
>Thread-3511::DEBUG::2012-12-09 13:23:12,324::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3511::DEBUG::2012-12-09 13:23:12,324::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3511::DEBUG::2012-12-09 13:23:12,324::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3511::DEBUG::2012-12-09 13:23:12,324::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3511::DEBUG::2012-12-09 13:23:12,324::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3511::DEBUG::2012-12-09 13:23:12,325::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3511::DEBUG::2012-12-09 13:23:12,325::task::978::TaskManager.Task::(_decref) Task=`9a800c7c-fb2b-4d2b-87a0-3e3773be2fe9`::ref 0 aborting False
>Dummy-321::DEBUG::2012-12-09 13:23:14,138::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:14,184::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193144 s, 53.0 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:23:15,309::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:15,318::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000415938 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-321::DEBUG::2012-12-09 13:23:16,190::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:16,242::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241807 s, 42.3 MB/s\n'; <rc> = 0
>Thread-3514::DEBUG::2012-12-09 13:23:17,536::task::588::TaskManager.Task::(_updateState) Task=`fc004a97-3c59-468e-96d8-3d2150e394fe`::moving from state init -> state preparing
>Thread-3514::INFO::2012-12-09 13:23:17,537::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3514::INFO::2012-12-09 13:23:17,537::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102360248566', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.6', 'code': 358, 'valid': False}}
>Thread-3514::DEBUG::2012-12-09 13:23:17,538::task::1172::TaskManager.Task::(prepare) Task=`fc004a97-3c59-468e-96d8-3d2150e394fe`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0102360248566', 'lastCheck': '2.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.6', 'code': 358, 'valid': False}}
>Thread-3514::DEBUG::2012-12-09 13:23:17,538::task::588::TaskManager.Task::(_updateState) Task=`fc004a97-3c59-468e-96d8-3d2150e394fe`::moving from state preparing -> state finished
>Thread-3514::DEBUG::2012-12-09 13:23:17,539::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3514::DEBUG::2012-12-09 13:23:17,539::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3514::DEBUG::2012-12-09 13:23:17,540::task::978::TaskManager.Task::(_decref) Task=`fc004a97-3c59-468e-96d8-3d2150e394fe`::ref 0 aborting False
>Thread-3516::DEBUG::2012-12-09 13:23:17,895::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3516::DEBUG::2012-12-09 13:23:17,895::task::588::TaskManager.Task::(_updateState) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::moving from state init -> state preparing
>Thread-3516::INFO::2012-12-09 13:23:17,895::logUtils::37::dispatcher::(wrapper) Run and protect: removeVM(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', vmUUID='c204995f-589e-4110-97ef-76494505123a', sdUUID=None, options=None)
>Thread-3516::DEBUG::2012-12-09 13:23:17,896::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a33bda96-2569-4eca-aabe-b913e3eff242`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3516::DEBUG::2012-12-09 13:23:17,896::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3516::DEBUG::2012-12-09 13:23:17,896::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3516::DEBUG::2012-12-09 13:23:17,896::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a33bda96-2569-4eca-aabe-b913e3eff242`::Granted request
>Thread-3516::DEBUG::2012-12-09 13:23:17,897::task::817::TaskManager.Task::(resourceAcquired) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3516::DEBUG::2012-12-09 13:23:17,897::task::978::TaskManager.Task::(_decref) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::ref 1 aborting False
>Thread-3516::INFO::2012-12-09 13:23:17,897::sp::1300::Storage.StoragePool::(removeVM) spUUID=4372535e-3f4d-11e2-8ba0-525400202852 vmUUID=c204995f-589e-4110-97ef-76494505123a sdUUID=None
>Thread-3516::INFO::2012-12-09 13:23:17,938::logUtils::39::dispatcher::(wrapper) Run and protect: removeVM, Return response: None
>Thread-3516::DEBUG::2012-12-09 13:23:17,939::task::1172::TaskManager.Task::(prepare) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::finished: None
>Thread-3516::DEBUG::2012-12-09 13:23:17,939::task::588::TaskManager.Task::(_updateState) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::moving from state preparing -> state finished
>Thread-3516::DEBUG::2012-12-09 13:23:17,940::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3516::DEBUG::2012-12-09 13:23:17,940::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3516::DEBUG::2012-12-09 13:23:17,941::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3516::DEBUG::2012-12-09 13:23:17,941::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3516::DEBUG::2012-12-09 13:23:17,942::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3516::DEBUG::2012-12-09 13:23:17,942::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3516::DEBUG::2012-12-09 13:23:17,943::task::978::TaskManager.Task::(_decref) Task=`8b258d0b-ca42-4b26-9df1-407acfd27091`::ref 0 aborting False
>Thread-3517::DEBUG::2012-12-09 13:23:18,008::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3517::DEBUG::2012-12-09 13:23:18,009::task::588::TaskManager.Task::(_updateState) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::moving from state init -> state preparing
>Thread-3517::INFO::2012-12-09 13:23:18,009::logUtils::37::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', spUUID='4372535e-3f4d-11e2-8ba0-525400202852', imgUUID='3a661f02-41b6-42ff-aa89-418f0f7f54cb', postZero='false', force='false')
>Thread-3517::INFO::2012-12-09 13:23:18,010::blockSD::810::Storage.StorageDomain::(validate) sdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-3517::DEBUG::2012-12-09 13:23:18,010::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3517::DEBUG::2012-12-09 13:23:18,118::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3517::DEBUG::2012-12-09 13:23:18,120::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,120::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,121::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,121::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,122::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,122::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3517::DEBUG::2012-12-09 13:23:18,230::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3517::DEBUG::2012-12-09 13:23:18,231::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,231::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', 'POOL_SPM_ID=2', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_LVER=49', '_SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850']
>Thread-3517::DEBUG::2012-12-09 13:23:18,232::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb`ReqID=`180e6f30-3c9e-44e9-a067-c745069156f9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3517::DEBUG::2012-12-09 13:23:18,233::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb' for lock type 'exclusive'
>Thread-3517::DEBUG::2012-12-09 13:23:18,233::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb' is free. Now locking as 'exclusive' (1 active user)
>Thread-3517::DEBUG::2012-12-09 13:23:18,233::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb`ReqID=`180e6f30-3c9e-44e9-a067-c745069156f9`::Granted request
>Thread-3517::DEBUG::2012-12-09 13:23:18,233::task::817::TaskManager.Task::(resourceAcquired) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::_resourcesAcquired: Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb (exclusive)
>Thread-3517::DEBUG::2012-12-09 13:23:18,234::task::978::TaskManager.Task::(_decref) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::ref 1 aborting False
>Thread-3517::DEBUG::2012-12-09 13:23:18,234::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`a9f9969a-e161-49dc-93cc-2dd1a7eb33bd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3517::DEBUG::2012-12-09 13:23:18,234::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' for lock type 'shared'
>Thread-3517::DEBUG::2012-12-09 13:23:18,234::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free. Now locking as 'shared' (1 active user)
>Thread-3517::DEBUG::2012-12-09 13:23:18,234::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895`ReqID=`a9f9969a-e161-49dc-93cc-2dd1a7eb33bd`::Granted request
>Thread-3517::DEBUG::2012-12-09 13:23:18,235::task::817::TaskManager.Task::(resourceAcquired) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::_resourcesAcquired: Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (shared)
>Thread-3517::DEBUG::2012-12-09 13:23:18,235::task::978::TaskManager.Task::(_decref) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::ref 1 aborting False
>Thread-3517::DEBUG::2012-12-09 13:23:18,235::lvm::393::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,236::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:18,248::__init__::1164::Storage.Misc.excCmd::(_log) 'dd if=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-321::DEBUG::2012-12-09 13:23:18,287::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164707 s, 62.2 MB/s\n'; <rc> = 0
>Thread-3517::DEBUG::2012-12-09 13:23:18,333::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3517::DEBUG::2012-12-09 13:23:18,339::lvm::422::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,339::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvremove --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " -f --autobackup n c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/38c6b100-31f8-4914-a760-eb1c5e15cce4' (cwd None)
>Thread-3517::DEBUG::2012-12-09 13:23:18,530::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '  WARNING: This metadata update is NOT backed up\n'; <rc> = 0
>Thread-3517::DEBUG::2012-12-09 13:23:18,531::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3517::DEBUG::2012-12-09 13:23:18,531::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3517::WARNING::2012-12-09 13:23:18,532::blockSD::925::Storage.StorageDomain::(rmDCVolLinks) Can't unlink /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/images/3a661f02-41b6-42ff-aa89-418f0f7f54cb/38c6b100-31f8-4914-a760-eb1c5e15cce4. [Errno 2] No such file or directory: '/rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/images/3a661f02-41b6-42ff-aa89-418f0f7f54cb/38c6b100-31f8-4914-a760-eb1c5e15cce4'
>Thread-3517::DEBUG::2012-12-09 13:23:18,532::blockSD::928::Storage.StorageDomain::(rmDCVolLinks) removed: []
>Thread-3517::ERROR::2012-12-09 13:23:18,533::task::853::TaskManager.Task::(_setError) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 861, in _run
>    return fn(*args, **kargs)
>  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/storage/hsm.py", line 1349, in deleteImage
>    dom.deleteImage(sdUUID, imgUUID, volsByImg)
>  File "/usr/share/vdsm/storage/blockSD.py", line 946, in deleteImage
>    self.rmDCImgDir(imgUUID, volsImgs)
>  File "/usr/share/vdsm/storage/blockSD.py", line 937, in rmDCImgDir
>    self.log.warning("Can't rmdir %s. %s", imgPath, exc_info = True)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 1068, in warning
>    self._log(WARNING, msg, args, **kwargs)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 1173, in _log
>    self.handle(record)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 1183, in handle
>    self.callHandlers(record)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 1220, in callHandlers
>    hdlr.handle(record)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 679, in handle
>    self.emit(record)
>  File "/usr/lib64/python2.6/logging/handlers.py", line 776, in emit
>    msg = self.format(record)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 654, in format
>    return fmt.format(record)
>  File "/usr/lib64/python2.6/logging/__init__.py", line 436, in format
>    record.message = record.getMessage()
>  File "/usr/lib64/python2.6/logging/__init__.py", line 306, in getMessage
>    msg = msg % self.args
>TypeError: not enough arguments for format string
>Thread-3517::DEBUG::2012-12-09 13:23:18,679::task::872::TaskManager.Task::(_run) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::Task._run: 62f708b5-d32a-4245-b413-25666be54fc6 ('c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', '4372535e-3f4d-11e2-8ba0-525400202852', '3a661f02-41b6-42ff-aa89-418f0f7f54cb', 'false', 'false') {} failed - stopping task
>Thread-3517::DEBUG::2012-12-09 13:23:18,679::task::1199::TaskManager.Task::(stop) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::stopping in state preparing (force False)
>Thread-3517::DEBUG::2012-12-09 13:23:18,680::task::978::TaskManager.Task::(_decref) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::ref 1 aborting True
>Thread-3517::INFO::2012-12-09 13:23:18,680::task::1157::TaskManager.Task::(prepare) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::aborting: Task is aborted: u'not enough arguments for format string' - code 100
>Thread-3517::DEBUG::2012-12-09 13:23:18,681::task::1162::TaskManager.Task::(prepare) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::Prepare: aborted: not enough arguments for format string
>Thread-3517::DEBUG::2012-12-09 13:23:18,681::task::978::TaskManager.Task::(_decref) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::ref 0 aborting True
>Thread-3517::DEBUG::2012-12-09 13:23:18,682::task::913::TaskManager.Task::(_doAbort) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::Task._doAbort: force False
>Thread-3517::DEBUG::2012-12-09 13:23:18,682::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3517::DEBUG::2012-12-09 13:23:18,683::task::588::TaskManager.Task::(_updateState) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::moving from state preparing -> state aborting
>Thread-3517::DEBUG::2012-12-09 13:23:18,683::task::537::TaskManager.Task::(__state_aborting) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::_aborting: recover policy none
>Thread-3517::DEBUG::2012-12-09 13:23:18,683::task::588::TaskManager.Task::(_updateState) Task=`62f708b5-d32a-4245-b413-25666be54fc6`::moving from state aborting -> state failed
>Thread-3517::DEBUG::2012-12-09 13:23:18,684::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb': < ResourceRef 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb', isValid: 'True' obj: 'None'>, 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': < ResourceRef 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', isValid: 'True' obj: 'None'>}
>Thread-3517::DEBUG::2012-12-09 13:23:18,684::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3517::DEBUG::2012-12-09 13:23:18,684::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb'
>Thread-3517::DEBUG::2012-12-09 13:23:18,684::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb' (0 active users)
>Thread-3517::DEBUG::2012-12-09 13:23:18,684::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb' is free, finding out if anyone is waiting for it.
>Thread-3517::DEBUG::2012-12-09 13:23:18,685::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.3a661f02-41b6-42ff-aa89-418f0f7f54cb', Clearing records.
>Thread-3517::DEBUG::2012-12-09 13:23:18,685::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895'
>Thread-3517::DEBUG::2012-12-09 13:23:18,685::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (0 active users)
>Thread-3517::DEBUG::2012-12-09 13:23:18,685::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' is free, finding out if anyone is waiting for it.
>Thread-3517::DEBUG::2012-12-09 13:23:18,685::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', Clearing records.
>Thread-3517::ERROR::2012-12-09 13:23:18,686::dispatcher::69::Storage.Dispatcher.Protect::(run) not enough arguments for format string
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 61, in run
>    result = ctask.prepare(self.func, *args, **kwargs)
>  File "/usr/share/vdsm/storage/task.py", line 1164, in prepare
>    raise self.error
>TypeError: not enough arguments for format string
>Thread-3518::DEBUG::2012-12-09 13:23:18,733::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3518::DEBUG::2012-12-09 13:23:18,734::task::588::TaskManager.Task::(_updateState) Task=`6b356c7a-1fda-4009-85a8-2e36e39ef383`::moving from state init -> state preparing
>Thread-3518::INFO::2012-12-09 13:23:18,735::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3518::INFO::2012-12-09 13:23:18,735::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3518::DEBUG::2012-12-09 13:23:18,736::task::1172::TaskManager.Task::(prepare) Task=`6b356c7a-1fda-4009-85a8-2e36e39ef383`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 49}}
>Thread-3518::DEBUG::2012-12-09 13:23:18,736::task::588::TaskManager.Task::(_updateState) Task=`6b356c7a-1fda-4009-85a8-2e36e39ef383`::moving from state preparing -> state finished
>Thread-3518::DEBUG::2012-12-09 13:23:18,737::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3518::DEBUG::2012-12-09 13:23:18,737::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3518::DEBUG::2012-12-09 13:23:18,737::task::978::TaskManager.Task::(_decref) Task=`6b356c7a-1fda-4009-85a8-2e36e39ef383`::ref 0 aborting False
>Thread-3519::DEBUG::2012-12-09 13:23:18,780::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3519::DEBUG::2012-12-09 13:23:18,781::task::588::TaskManager.Task::(_updateState) Task=`4bb04f43-1a19-4c80-9331-55debc841c83`::moving from state init -> state preparing
>Thread-3519::INFO::2012-12-09 13:23:18,781::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None)
>Thread-3519::DEBUG::2012-12-09 13:23:18,782::taskManager::103::TaskManager::(getAllTasksStatuses) Entry.
>Thread-3519::DEBUG::2012-12-09 13:23:18,782::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-3519::INFO::2012-12-09 13:23:18,782::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}}
>Thread-3519::DEBUG::2012-12-09 13:23:18,783::task::1172::TaskManager.Task::(prepare) Task=`4bb04f43-1a19-4c80-9331-55debc841c83`::finished: {'allTasksStatus': {}}
>Thread-3519::DEBUG::2012-12-09 13:23:18,783::task::588::TaskManager.Task::(_updateState) Task=`4bb04f43-1a19-4c80-9331-55debc841c83`::moving from state preparing -> state finished
>Thread-3519::DEBUG::2012-12-09 13:23:18,783::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3519::DEBUG::2012-12-09 13:23:18,784::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3519::DEBUG::2012-12-09 13:23:18,784::task::978::TaskManager.Task::(_decref) Task=`4bb04f43-1a19-4c80-9331-55debc841c83`::ref 0 aborting False
>Thread-3520::DEBUG::2012-12-09 13:23:18,798::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3520::DEBUG::2012-12-09 13:23:18,798::task::588::TaskManager.Task::(_updateState) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::moving from state init -> state preparing
>Thread-3520::INFO::2012-12-09 13:23:18,799::logUtils::37::dispatcher::(wrapper) Run and protect: spmStop(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3520::DEBUG::2012-12-09 13:23:18,800::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5c0bbb18-6ed9-42f9-90d3-c443281629a9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3520::DEBUG::2012-12-09 13:23:18,800::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-3520::DEBUG::2012-12-09 13:23:18,801::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-3520::DEBUG::2012-12-09 13:23:18,801::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`5c0bbb18-6ed9-42f9-90d3-c443281629a9`::Granted request
>Thread-3520::DEBUG::2012-12-09 13:23:18,802::task::817::TaskManager.Task::(resourceAcquired) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (exclusive)
>Thread-3520::DEBUG::2012-12-09 13:23:18,802::task::978::TaskManager.Task::(_decref) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::ref 1 aborting False
>Thread-3520::DEBUG::2012-12-09 13:23:18,803::sp::323::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process
>Thread-3520::DEBUG::2012-12-09 13:23:18,803::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6058f0e0-0044-478b-b837-1d6df8367b67`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3520::DEBUG::2012-12-09 13:23:18,803::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-3520::DEBUG::2012-12-09 13:23:18,804::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-3520::DEBUG::2012-12-09 13:23:18,804::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`6058f0e0-0044-478b-b837-1d6df8367b67`::Granted request
>Thread-3520::DEBUG::2012-12-09 13:23:18,804::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141`ReqID=`d7e40e1e-097b-4cc0-a418-f91cebec14aa`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3520::DEBUG::2012-12-09 13:23:18,804::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141' for lock type 'exclusive'
>Thread-3520::DEBUG::2012-12-09 13:23:18,805::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141' is free. Now locking as 'exclusive' (1 active user)
>Thread-3520::DEBUG::2012-12-09 13:23:18,805::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141`ReqID=`d7e40e1e-097b-4cc0-a418-f91cebec14aa`::Granted request
>Thread-3520::DEBUG::2012-12-09 13:23:18,805::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141'
>Thread-3520::DEBUG::2012-12-09 13:23:18,805::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141' (0 active users)
>Thread-3520::DEBUG::2012-12-09 13:23:18,805::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141' is free, finding out if anyone is waiting for it.
>Thread-3520::DEBUG::2012-12-09 13:23:18,806::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c080b7a1-1d5f-449f-9a65-375961d37141', Clearing records.
>Thread-3520::DEBUG::2012-12-09 13:23:18,806::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3520::DEBUG::2012-12-09 13:23:18,806::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3520::DEBUG::2012-12-09 13:23:18,806::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3520::DEBUG::2012-12-09 13:23:18,806::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3520::DEBUG::2012-12-09 13:23:18,808::sp::357::Storage.StoragePool::(cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master
>Thread-3520::DEBUG::2012-12-09 13:23:18,843::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/umount /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/master' (cwd None)
>Thread-3520::DEBUG::2012-12-09 13:23:19,218::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction
>Thread-3520::DEBUG::2012-12-09 13:23:19,219::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes
>Thread-3520::DEBUG::2012-12-09 13:23:19,220::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=49', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'ROLE=Master', 'SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', '_SHA_CKSUM=f28d92826c04f2eaec1d2af84d2c5e2d334eec17']
>Thread-3520::DEBUG::2012-12-09 13:23:19,220::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,225::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,225::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,225::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,225::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,226::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3520::DEBUG::2012-12-09 13:23:19,817::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3520::DEBUG::2012-12-09 13:23:19,818::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:19,818::blockSD::297::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=f28d92826c04f2eaec1d2af84d2c5e2d334eec17 removing=MDT__SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850, MDT_POOL_SPM_ID=2
>Thread-3520::DEBUG::2012-12-09 13:23:19,819::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=2 --deltag MDT__SHA_CKSUM=743c4c628b29dda1560dc3903f03dd4e45868850 --addtag MDT_POOL_SPM_ID=-1 --addtag MDT__SHA_CKSUM=f28d92826c04f2eaec1d2af84d2c5e2d334eec17 c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3520::DEBUG::2012-12-09 13:23:20,137::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3520::DEBUG::2012-12-09 13:23:20,138::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:20,138::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3520::DEBUG::2012-12-09 13:23:20,139::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction
>Thread-3520::INFO::2012-12-09 13:23:20,139::safelease::239::SANLock::(release) Releasing cluster lock for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Dummy-321::INFO::2012-12-09 13:23:20,471::storage_mailbox::691::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped
>Thread-3520::DEBUG::2012-12-09 13:23:20,477::safelease::250::SANLock::(release) Cluster lock for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 successfully released
>Thread-3520::INFO::2012-12-09 13:23:20,478::logUtils::39::dispatcher::(wrapper) Run and protect: spmStop, Return response: None
>Thread-3520::DEBUG::2012-12-09 13:23:20,478::task::1172::TaskManager.Task::(prepare) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::finished: None
>Thread-3520::DEBUG::2012-12-09 13:23:20,478::task::588::TaskManager.Task::(_updateState) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::moving from state preparing -> state finished
>Thread-3520::DEBUG::2012-12-09 13:23:20,478::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3520::DEBUG::2012-12-09 13:23:20,479::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3520::DEBUG::2012-12-09 13:23:20,479::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3520::DEBUG::2012-12-09 13:23:20,479::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3520::DEBUG::2012-12-09 13:23:20,479::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3520::DEBUG::2012-12-09 13:23:20,479::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3520::DEBUG::2012-12-09 13:23:20,480::task::978::TaskManager.Task::(_decref) Task=`0c56069e-7271-4e78-ac87-7117d69d7a16`::ref 0 aborting False
>Thread-3522::DEBUG::2012-12-09 13:23:20,635::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3522::DEBUG::2012-12-09 13:23:20,636::task::588::TaskManager.Task::(_updateState) Task=`7e31b2c6-5083-403b-bd9c-35202d945a24`::moving from state init -> state preparing
>Thread-3522::INFO::2012-12-09 13:23:20,637::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', options=None)
>Thread-3522::INFO::2012-12-09 13:23:20,637::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 49}}
>Thread-3522::DEBUG::2012-12-09 13:23:20,638::task::1172::TaskManager.Task::(prepare) Task=`7e31b2c6-5083-403b-bd9c-35202d945a24`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 49}}
>Thread-3522::DEBUG::2012-12-09 13:23:20,638::task::588::TaskManager.Task::(_updateState) Task=`7e31b2c6-5083-403b-bd9c-35202d945a24`::moving from state preparing -> state finished
>Thread-3522::DEBUG::2012-12-09 13:23:20,638::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3522::DEBUG::2012-12-09 13:23:20,638::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3522::DEBUG::2012-12-09 13:23:20,639::task::978::TaskManager.Task::(_decref) Task=`7e31b2c6-5083-403b-bd9c-35202d945a24`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:23:21,905::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:21,905::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:22,008::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:22,009::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:22,009::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:22,009::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:23:25,321::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:23:25,323::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:26,436::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:23:26,437::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:23:26,437::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:26,446::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000474111 s, 8.6 MB/s\n'; <rc> = 0
>Thread-3526::DEBUG::2012-12-09 13:23:27,941::task::588::TaskManager.Task::(_updateState) Task=`404dd281-c0db-47a6-8b25-b26a0c34e322`::moving from state init -> state preparing
>Thread-3526::INFO::2012-12-09 13:23:27,942::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3526::INFO::2012-12-09 13:23:27,942::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101220607758', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-3526::DEBUG::2012-12-09 13:23:27,943::task::1172::TaskManager.Task::(prepare) Task=`404dd281-c0db-47a6-8b25-b26a0c34e322`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0101220607758', 'lastCheck': '1.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '5.9', 'code': 358, 'valid': False}}
>Thread-3526::DEBUG::2012-12-09 13:23:27,943::task::588::TaskManager.Task::(_updateState) Task=`404dd281-c0db-47a6-8b25-b26a0c34e322`::moving from state preparing -> state finished
>Thread-3526::DEBUG::2012-12-09 13:23:27,943::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3526::DEBUG::2012-12-09 13:23:27,944::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3526::DEBUG::2012-12-09 13:23:27,944::task::978::TaskManager.Task::(_decref) Task=`404dd281-c0db-47a6-8b25-b26a0c34e322`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:23:32,010::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:32,012::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:32,119::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:32,120::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:32,121::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:32,121::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:23:36,450::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:36,460::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00053463 s, 7.7 MB/s\n'; <rc> = 0
>Thread-3532::DEBUG::2012-12-09 13:23:38,590::task::588::TaskManager.Task::(_updateState) Task=`0ce847cd-caba-4420-9694-fbb81b38693c`::moving from state init -> state preparing
>Thread-3532::INFO::2012-12-09 13:23:38,590::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3532::INFO::2012-12-09 13:23:38,591::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113389492035', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3532::DEBUG::2012-12-09 13:23:38,591::task::1172::TaskManager.Task::(prepare) Task=`0ce847cd-caba-4420-9694-fbb81b38693c`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0113389492035', 'lastCheck': '2.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False}}
>Thread-3532::DEBUG::2012-12-09 13:23:38,592::task::588::TaskManager.Task::(_updateState) Task=`0ce847cd-caba-4420-9694-fbb81b38693c`::moving from state preparing -> state finished
>Thread-3532::DEBUG::2012-12-09 13:23:38,592::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3532::DEBUG::2012-12-09 13:23:38,593::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3532::DEBUG::2012-12-09 13:23:38,593::task::978::TaskManager.Task::(_decref) Task=`0ce847cd-caba-4420-9694-fbb81b38693c`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:23:42,122::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:42,123::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:42,235::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:42,236::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:42,236::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:42,236::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:23:46,464::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:46,478::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000602406 s, 6.8 MB/s\n'; <rc> = 0
>Thread-3538::DEBUG::2012-12-09 13:23:49,483::task::588::TaskManager.Task::(_updateState) Task=`9486792e-b6ef-4e44-a02e-ec08adcb3ed1`::moving from state init -> state preparing
>Thread-3538::INFO::2012-12-09 13:23:49,484::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3538::INFO::2012-12-09 13:23:49,484::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149040222168', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.2', 'code': 358, 'valid': False}}
>Thread-3538::DEBUG::2012-12-09 13:23:49,484::task::1172::TaskManager.Task::(prepare) Task=`9486792e-b6ef-4e44-a02e-ec08adcb3ed1`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0149040222168', 'lastCheck': '3.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '7.2', 'code': 358, 'valid': False}}
>Thread-3538::DEBUG::2012-12-09 13:23:49,485::task::588::TaskManager.Task::(_updateState) Task=`9486792e-b6ef-4e44-a02e-ec08adcb3ed1`::moving from state preparing -> state finished
>Thread-3538::DEBUG::2012-12-09 13:23:49,485::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3538::DEBUG::2012-12-09 13:23:49,485::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3538::DEBUG::2012-12-09 13:23:49,485::task::978::TaskManager.Task::(_decref) Task=`9486792e-b6ef-4e44-a02e-ec08adcb3ed1`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:23:52,237::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:23:52,238::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:23:52,348::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:23:52,350::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:23:52,350::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:23:52,351::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:23:56,482::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:23:56,489::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000443113 s, 9.2 MB/s\n'; <rc> = 0
>Thread-3544::DEBUG::2012-12-09 13:24:00,329::task::588::TaskManager.Task::(_updateState) Task=`c58f3fd3-c97a-4f20-bd7b-d702321e09e6`::moving from state init -> state preparing
>Thread-3544::INFO::2012-12-09 13:24:00,329::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3544::INFO::2012-12-09 13:24:00,330::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00756406784058', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.0', 'code': 358, 'valid': False}}
>Thread-3544::DEBUG::2012-12-09 13:24:00,330::task::1172::TaskManager.Task::(prepare) Task=`c58f3fd3-c97a-4f20-bd7b-d702321e09e6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.00756406784058', 'lastCheck': '3.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '8.0', 'code': 358, 'valid': False}}
>Thread-3544::DEBUG::2012-12-09 13:24:00,331::task::588::TaskManager.Task::(_updateState) Task=`c58f3fd3-c97a-4f20-bd7b-d702321e09e6`::moving from state preparing -> state finished
>Thread-3544::DEBUG::2012-12-09 13:24:00,331::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3544::DEBUG::2012-12-09 13:24:00,332::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3544::DEBUG::2012-12-09 13:24:00,332::task::978::TaskManager.Task::(_decref) Task=`c58f3fd3-c97a-4f20-bd7b-d702321e09e6`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:24:02,353::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:02,354::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:02,473::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:24:02,474::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:24:02,474::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:24:02,475::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:24:06,491::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:06,502::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000773079 s, 5.3 MB/s\n'; <rc> = 0
>Thread-3550::DEBUG::2012-12-09 13:24:12,439::task::588::TaskManager.Task::(_updateState) Task=`ae6fe357-e704-4f36-9eaa-fada7bf89b00`::moving from state init -> state preparing
>Thread-3550::INFO::2012-12-09 13:24:12,440::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3550::INFO::2012-12-09 13:24:12,441::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108799934387', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-3550::DEBUG::2012-12-09 13:24:12,441::task::1172::TaskManager.Task::(prepare) Task=`ae6fe357-e704-4f36-9eaa-fada7bf89b00`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0108799934387', 'lastCheck': '5.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.0', 'code': 358, 'valid': False}}
>Thread-3550::DEBUG::2012-12-09 13:24:12,442::task::588::TaskManager.Task::(_updateState) Task=`ae6fe357-e704-4f36-9eaa-fada7bf89b00`::moving from state preparing -> state finished
>Thread-3550::DEBUG::2012-12-09 13:24:12,442::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3550::DEBUG::2012-12-09 13:24:12,442::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3550::DEBUG::2012-12-09 13:24:12,442::task::978::TaskManager.Task::(_decref) Task=`ae6fe357-e704-4f36-9eaa-fada7bf89b00`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:24:12,476::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:12,477::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:12,580::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:24:12,581::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:24:12,582::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::ERROR::2012-12-09 13:24:12,582::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain c080b7a1-1d5f-449f-9a65-375961d37141 monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'c080b7a1-1d5f-449f-9a65-375961d37141',)
>Thread-50::DEBUG::2012-12-09 13:24:16,503::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::DEBUG::2012-12-09 13:24:16,504::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:24:16,505::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:16,511::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000378526 s, 5.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:24:16,511::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000378526 s, 5.4 MB/s'], size: 2048
>Thread-50::DEBUG::2012-12-09 13:24:16,511::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-50::WARNING::2012-12-09 13:24:16,512::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-50::DEBUG::2012-12-09 13:24:16,512::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-50::DEBUG::2012-12-09 13:24:16,512::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,512::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,512::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,513::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,513::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,513::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:16,617::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:24:16,618::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-50::DEBUG::2012-12-09 13:24:16,619::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=1', '_SHA_CKSUM=f52314b61dc9cceaf0636eb85603f91519e54b21', 'POOL_SPM_LVER=50']
>Thread-50::WARNING::2012-12-09 13:24:16,620::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-50::WARNING::2012-12-09 13:24:16,620::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-50::WARNING::2012-12-09 13:24:16,620::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:24:16,621::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:16,718::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:24:16,719::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:16,728::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00048006 s, 8.5 MB/s\n'; <rc> = 0
>Thread-3555::DEBUG::2012-12-09 13:24:19,384::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3555::DEBUG::2012-12-09 13:24:19,385::task::588::TaskManager.Task::(_updateState) Task=`6d7e4390-9570-46fc-9276-24613332339c`::moving from state init -> state preparing
>Thread-3555::INFO::2012-12-09 13:24:19,385::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'mgr.rhev.icp.selfip.net:/var/lib/exports/iso', 'iqn': '', 'portal': '', 'user': '', 'protocol_version': '4', 'password': '******', 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}], options=None)
>Thread-3555::INFO::2012-12-09 13:24:19,385::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3555::DEBUG::2012-12-09 13:24:19,386::task::1172::TaskManager.Task::(prepare) Task=`6d7e4390-9570-46fc-9276-24613332339c`::finished: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3555::DEBUG::2012-12-09 13:24:19,386::task::588::TaskManager.Task::(_updateState) Task=`6d7e4390-9570-46fc-9276-24613332339c`::moving from state preparing -> state finished
>Thread-3555::DEBUG::2012-12-09 13:24:19,386::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3555::DEBUG::2012-12-09 13:24:19,387::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3555::DEBUG::2012-12-09 13:24:19,387::task::978::TaskManager.Task::(_decref) Task=`6d7e4390-9570-46fc-9276-24613332339c`::ref 0 aborting False
>Thread-3556::DEBUG::2012-12-09 13:24:19,417::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3556::DEBUG::2012-12-09 13:24:19,418::task::588::TaskManager.Task::(_updateState) Task=`ed14b57c-5ff1-4ae2-b51d-147bce0a9e74`::moving from state init -> state preparing
>Thread-3556::INFO::2012-12-09 13:24:19,418::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'mgr.rhev.icp.selfip.net:/var/lib/exports/iso', 'iqn': '', 'portal': '', 'user': '', 'protocol_version': '4', 'password': '******', 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}], options=None)
>Thread-3556::DEBUG::2012-12-09 13:24:19,419::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=4 mgr.rhev.icp.selfip.net:/var/lib/exports/iso /rhev/data-center/mnt/mgr.rhev.icp.selfip.net:_var_lib_exports_iso' (cwd None)
>Thread-3556::INFO::2012-12-09 13:24:19,760::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3556::DEBUG::2012-12-09 13:24:19,760::task::1172::TaskManager.Task::(prepare) Task=`ed14b57c-5ff1-4ae2-b51d-147bce0a9e74`::finished: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3556::DEBUG::2012-12-09 13:24:19,761::task::588::TaskManager.Task::(_updateState) Task=`ed14b57c-5ff1-4ae2-b51d-147bce0a9e74`::moving from state preparing -> state finished
>Thread-3556::DEBUG::2012-12-09 13:24:19,761::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3556::DEBUG::2012-12-09 13:24:19,761::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3556::DEBUG::2012-12-09 13:24:19,761::task::978::TaskManager.Task::(_decref) Task=`ed14b57c-5ff1-4ae2-b51d-147bce0a9e74`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:24:22,583::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-51::DEBUG::2012-12-09 13:24:22,584::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-51::DEBUG::2012-12-09 13:24:22,584::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-51::DEBUG::2012-12-09 13:24:22,585::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-51::DEBUG::2012-12-09 13:24:22,585::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:22,604::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:24:22,605::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-51::DEBUG::2012-12-09 13:24:22,606::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>Thread-3558::DEBUG::2012-12-09 13:24:22,761::task::588::TaskManager.Task::(_updateState) Task=`bd77ff2c-08ce-4650-b527-184ff09b562d`::moving from state init -> state preparing
>Thread-3558::INFO::2012-12-09 13:24:22,761::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3558::INFO::2012-12-09 13:24:22,762::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091769695282', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.2', 'code': 358, 'valid': False}}
>Thread-3558::DEBUG::2012-12-09 13:24:22,762::task::1172::TaskManager.Task::(prepare) Task=`bd77ff2c-08ce-4650-b527-184ff09b562d`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0091769695282', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0', 'lastCheck': '10.2', 'code': 358, 'valid': False}}
>Thread-3558::DEBUG::2012-12-09 13:24:22,763::task::588::TaskManager.Task::(_updateState) Task=`bd77ff2c-08ce-4650-b527-184ff09b562d`::moving from state preparing -> state finished
>Thread-3558::DEBUG::2012-12-09 13:24:22,763::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3558::DEBUG::2012-12-09 13:24:22,764::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3558::DEBUG::2012-12-09 13:24:22,764::task::978::TaskManager.Task::(_decref) Task=`bd77ff2c-08ce-4650-b527-184ff09b562d`::ref 0 aborting False
>Thread-3560::DEBUG::2012-12-09 13:24:23,283::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3560::DEBUG::2012-12-09 13:24:23,284::task::588::TaskManager.Task::(_updateState) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::moving from state init -> state preparing
>Thread-3560::INFO::2012-12-09 13:24:23,284::logUtils::37::dispatcher::(wrapper) Run and protect: refreshStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', msdUUID='c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', masterVersion=1, options=None)
>Thread-3560::DEBUG::2012-12-09 13:24:23,284::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`27749418-d8cd-4984-9537-43c5d11da181`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3560::DEBUG::2012-12-09 13:24:23,285::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'shared'
>Thread-3560::DEBUG::2012-12-09 13:24:23,285::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'shared' (1 active user)
>Thread-3560::DEBUG::2012-12-09 13:24:23,285::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`27749418-d8cd-4984-9537-43c5d11da181`::Granted request
>Thread-3560::DEBUG::2012-12-09 13:24:23,285::task::817::TaskManager.Task::(resourceAcquired) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (shared)
>Thread-3560::DEBUG::2012-12-09 13:24:23,286::task::978::TaskManager.Task::(_decref) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::ref 1 aborting False
>Thread-3560::INFO::2012-12-09 13:24:23,286::blockSD::810::Storage.StorageDomain::(validate) sdUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-3560::DEBUG::2012-12-09 13:24:23,286::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:23,392::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:23,393::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,394::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,394::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,395::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,395::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,395::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:23,492::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:23,493::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,493::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=1', '_SHA_CKSUM=f52314b61dc9cceaf0636eb85603f91519e54b21', 'POOL_SPM_LVER=50']
>Thread-3560::DEBUG::2012-12-09 13:24:23,494::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,494::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,494::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,494::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,495::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,495::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:23,495::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>MainProcess|Thread-51::DEBUG::2012-12-09 13:24:23,611::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-51::DEBUG::2012-12-09 13:24:23,614::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host2/scan' (cwd None)
>MainProcess|Thread-51::DEBUG::2012-12-09 13:24:23,617::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host3/scan' (cwd None)
>MainProcess|Thread-51::DEBUG::2012-12-09 13:24:23,621::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-51::DEBUG::2012-12-09 13:24:25,624::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:25,649::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-51::DEBUG::2012-12-09 13:24:25,650::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,650::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,651::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,651::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,652::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,652::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,653::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-3560::DEBUG::2012-12-09 13:24:25,653::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-51::DEBUG::2012-12-09 13:24:25,654::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:25,654::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-3560::DEBUG::2012-12-09 13:24:25,656::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3560::DEBUG::2012-12-09 13:24:25,656::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:25,662::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:25,681::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:25,681::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-3560::DEBUG::2012-12-09 13:24:25,689::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-3560::DEBUG::2012-12-09 13:24:25,691::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-3560::DEBUG::2012-12-09 13:24:25,694::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host2/scan' (cwd None)
>MainProcess|Thread-3560::DEBUG::2012-12-09 13:24:25,696::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host3/scan' (cwd None)
>MainProcess|Thread-3560::DEBUG::2012-12-09 13:24:25,699::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-51::DEBUG::2012-12-09 13:24:25,770::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:24:25,772::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:24:25,772::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:25,780::fileSD::124::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/mgr.rhev.icp.selfip.net:_var_lib_exports_iso/c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:24:25,781::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with FileMetadataRW backend
>Thread-51::DEBUG::2012-12-09 13:24:26,166::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=NFS_ISO_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=c34646d6-2dc7-11e2-8cb6-525400202852,4372535e-3f4d-11e2-8ba0-525400202852', 'REMOTE_PATH=mgr.rhev.icp.selfip.net:/var/lib/exports/iso', 'ROLE=Regular', 'SDUUID=c080b7a1-1d5f-449f-9a65-375961d37141', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=f0d093bcea4329aa26367556a4a04db827b67f62']
>Thread-51::DEBUG::2012-12-09 13:24:26,168::fileSD::447::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-51::DEBUG::2012-12-09 13:24:26,169::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'c080b7a1-1d5f-449f-9a65-375961d37141_imageNS'
>Thread-51::DEBUG::2012-12-09 13:24:26,169::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'c080b7a1-1d5f-449f-9a65-375961d37141_volumeNS'
>Thread-51::DEBUG::2012-12-09 13:24:26,174::domainMonitor::216::Storage.DomainMonitorThread::(_monitorDomain) Domain c080b7a1-1d5f-449f-9a65-375961d37141 changed its status to Valid
>Thread-3560::DEBUG::2012-12-09 13:24:27,702::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:27,736::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:27,737::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,738::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,738::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,739::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,739::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,740::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,740::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-3560::DEBUG::2012-12-09 13:24:27,741::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,743::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:27,854::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:27,855::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,856::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>Thread-3560::DEBUG::2012-12-09 13:24:27,856::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata count=1' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:27,863::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000654982 s, 3.1 MB/s\n'; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:27,863::misc::335::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000654982 s, 3.1 MB/s'], size: 2048
>Thread-3560::DEBUG::2012-12-09 13:24:27,864::persistentDict::226::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-3560::WARNING::2012-12-09 13:24:27,864::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-3560::DEBUG::2012-12-09 13:24:27,864::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>Thread-3560::DEBUG::2012-12-09 13:24:27,864::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,865::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,865::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,865::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,865::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,866::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:27,967::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:27,968::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,969::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=1', '_SHA_CKSUM=f52314b61dc9cceaf0636eb85603f91519e54b21', 'POOL_SPM_LVER=50']
>Thread-3560::DEBUG::2012-12-09 13:24:27,969::lvm::319::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:27,969::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:28,078::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:28,079::lvm::342::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::WARNING::2012-12-09 13:24:28,079::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_imageNS already registered
>Thread-3560::WARNING::2012-12-09 13:24:28,080::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_volumeNS already registered
>Thread-3560::WARNING::2012-12-09 13:24:28,080::blockSD::425::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c294c8b1-b2e7-4564-bbb2-2d6c1f54b895_lvmActivationNS already registered
>Thread-50::DEBUG::2012-12-09 13:24:28,081::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:28,089::sp::1539::Storage.StoragePool::(getMasterDomain) Master domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 verified, version 1
>Thread-3560::DEBUG::2012-12-09 13:24:28,089::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>Thread-3560::DEBUG::2012-12-09 13:24:28,090::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3560::DEBUG::2012-12-09 13:24:28,090::lvm::467::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,091::lvm::469::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,092::lvm::478::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,092::lvm::490::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,093::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,093::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c294c8b1-b2e7-4564-bbb2-2d6c1f54b895' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:28,183::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:24:28,184::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-3560::DEBUG::2012-12-09 13:24:28,193::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:28,195::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,196::persistentDict::226::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['SDUUID=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895', 'LOCKPOLICY=', 'TYPE=ISCSI', u'PV0=pv:1IET_00010001,uuid:VULudr-Qcdq-mNPL-CGFv-cDBn-3d31-2JAsaH,pestart:0,pecount:797,mapoffset:0', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'VGUUID=eemJjR-Qywf-fGpe-wa0U-G2Z3-KlXP-zgQIIM', 'PHYBLKSIZE=512', 'CLASS=Data', 'DESCRIPTION=ISCSI_DATA_0', 'VERSION=3', 'LEASETIMESEC=60', 'IOOPTIMEOUTSEC=10', 'POOL_DESCRIPTION=Default', 'ROLE=Master', 'POOL_UUID=4372535e-3f4d-11e2-8ba0-525400202852', 'MASTER_VERSION=1', u'POOL_DOMAINS=c294c8b1-b2e7-4564-bbb2-2d6c1f54b895:Active,c080b7a1-1d5f-449f-9a65-375961d37141:Active', 'POOL_SPM_ID=1', '_SHA_CKSUM=f52314b61dc9cceaf0636eb85603f91519e54b21', 'POOL_SPM_LVER=50']
>Thread-3560::DEBUG::2012-12-09 13:24:28,197::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-3560::WARNING::2012-12-09 13:24:28,198::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852 already exists
>Thread-3560::DEBUG::2012-12-09 13:24:28,199::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-3560::DEBUG::2012-12-09 13:24:28,200::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:28,206::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000532834 s, 7.7 MB/s\n'; <rc> = 0
>Thread-3560::DEBUG::2012-12-09 13:24:28,316::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-3560::WARNING::2012-12-09 13:24:28,317::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-3560::DEBUG::2012-12-09 13:24:28,317::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-3560::INFO::2012-12-09 13:24:28,319::sp::1133::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/mgr.rhev.icp.selfip.net:_var_lib_exports_iso/c080b7a1-1d5f-449f-9a65-375961d37141 to /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-3560::WARNING::2012-12-09 13:24:28,320::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/images already exists
>Thread-3560::WARNING::2012-12-09 13:24:28,320::fileUtils::181::fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/dom_md already exists
>Thread-3560::INFO::2012-12-09 13:24:28,320::sp::1133::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 to /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-3560::INFO::2012-12-09 13:24:28,321::sp::1133::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 to /rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd
>Thread-3560::INFO::2012-12-09 13:24:28,321::logUtils::39::dispatcher::(wrapper) Run and protect: refreshStoragePool, Return response: None
>Thread-3560::DEBUG::2012-12-09 13:24:28,321::task::1172::TaskManager.Task::(prepare) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::finished: None
>Thread-3560::DEBUG::2012-12-09 13:24:28,322::task::588::TaskManager.Task::(_updateState) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::moving from state preparing -> state finished
>Thread-3560::DEBUG::2012-12-09 13:24:28,322::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3560::DEBUG::2012-12-09 13:24:28,322::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3560::DEBUG::2012-12-09 13:24:28,322::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3560::DEBUG::2012-12-09 13:24:28,322::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3560::DEBUG::2012-12-09 13:24:28,323::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3560::DEBUG::2012-12-09 13:24:28,323::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3560::DEBUG::2012-12-09 13:24:28,323::task::978::TaskManager.Task::(_decref) Task=`615a5788-0b80-44ce-ab0b-ca4e2badfc74`::ref 0 aborting False
>Thread-3567::DEBUG::2012-12-09 13:24:33,140::task::588::TaskManager.Task::(_updateState) Task=`be81f241-2512-4b88-9d74-e2cc21878172`::moving from state init -> state preparing
>Thread-3567::INFO::2012-12-09 13:24:33,141::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3567::INFO::2012-12-09 13:24:33,141::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0220851898193', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00125789642334', 'lastCheck': '7.0', 'code': 0, 'valid': True}}
>Thread-3567::DEBUG::2012-12-09 13:24:33,142::task::1172::TaskManager.Task::(prepare) Task=`be81f241-2512-4b88-9d74-e2cc21878172`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0220851898193', 'lastCheck': '4.9', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00125789642334', 'lastCheck': '7.0', 'code': 0, 'valid': True}}
>Thread-3567::DEBUG::2012-12-09 13:24:33,142::task::588::TaskManager.Task::(_updateState) Task=`be81f241-2512-4b88-9d74-e2cc21878172`::moving from state preparing -> state finished
>Thread-3567::DEBUG::2012-12-09 13:24:33,142::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3567::DEBUG::2012-12-09 13:24:33,142::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3567::DEBUG::2012-12-09 13:24:33,142::task::978::TaskManager.Task::(_decref) Task=`be81f241-2512-4b88-9d74-e2cc21878172`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:24:38,208::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:38,223::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000910545 s, 4.5 MB/s\n'; <rc> = 0
>Thread-3573::DEBUG::2012-12-09 13:24:43,474::task::588::TaskManager.Task::(_updateState) Task=`17388e92-751f-4023-a786-ce53251db31e`::moving from state init -> state preparing
>Thread-3573::INFO::2012-12-09 13:24:43,474::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3573::INFO::2012-12-09 13:24:43,475::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160758495331', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00284004211426', 'lastCheck': '7.3', 'code': 0, 'valid': True}}
>Thread-3573::DEBUG::2012-12-09 13:24:43,475::task::1172::TaskManager.Task::(prepare) Task=`17388e92-751f-4023-a786-ce53251db31e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0160758495331', 'lastCheck': '5.2', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00284004211426', 'lastCheck': '7.3', 'code': 0, 'valid': True}}
>Thread-3573::DEBUG::2012-12-09 13:24:43,476::task::588::TaskManager.Task::(_updateState) Task=`17388e92-751f-4023-a786-ce53251db31e`::moving from state preparing -> state finished
>Thread-3573::DEBUG::2012-12-09 13:24:43,476::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3573::DEBUG::2012-12-09 13:24:43,477::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3573::DEBUG::2012-12-09 13:24:43,477::task::978::TaskManager.Task::(_decref) Task=`17388e92-751f-4023-a786-ce53251db31e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:24:48,228::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:48,243::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000761251 s, 5.4 MB/s\n'; <rc> = 0
>Thread-3579::DEBUG::2012-12-09 13:24:53,881::task::588::TaskManager.Task::(_updateState) Task=`81c3f1e6-44e7-4aa2-9f12-c88ef1a72291`::moving from state init -> state preparing
>Thread-3579::INFO::2012-12-09 13:24:53,882::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3579::INFO::2012-12-09 13:24:53,882::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163629055023', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00130915641785', 'lastCheck': '7.7', 'code': 0, 'valid': True}}
>Thread-3579::DEBUG::2012-12-09 13:24:53,882::task::1172::TaskManager.Task::(prepare) Task=`81c3f1e6-44e7-4aa2-9f12-c88ef1a72291`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163629055023', 'lastCheck': '5.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00130915641785', 'lastCheck': '7.7', 'code': 0, 'valid': True}}
>Thread-3579::DEBUG::2012-12-09 13:24:53,882::task::588::TaskManager.Task::(_updateState) Task=`81c3f1e6-44e7-4aa2-9f12-c88ef1a72291`::moving from state preparing -> state finished
>Thread-3579::DEBUG::2012-12-09 13:24:53,883::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3579::DEBUG::2012-12-09 13:24:53,883::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3579::DEBUG::2012-12-09 13:24:53,883::task::978::TaskManager.Task::(_decref) Task=`81c3f1e6-44e7-4aa2-9f12-c88ef1a72291`::ref 0 aborting False
>Thread-51::DEBUG::2012-12-09 13:24:56,187::domainMonitor::170::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:24:56,187::lvm::352::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:56,188::__init__::1164::Storage.Misc.excCmd::(_log) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%1IET_00010001%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free c080b7a1-1d5f-449f-9a65-375961d37141' (cwd None)
>Thread-51::DEBUG::2012-12-09 13:24:56,297::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = '  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found\n'; <rc> = 5
>Thread-51::WARNING::2012-12-09 13:24:56,299::lvm::356::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "c080b7a1-1d5f-449f-9a65-375961d37141" not found']
>Thread-51::DEBUG::2012-12-09 13:24:56,299::lvm::379::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-51::DEBUG::2012-12-09 13:24:56,306::fileSD::124::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/mgr.rhev.icp.selfip.net:_var_lib_exports_iso/c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-51::DEBUG::2012-12-09 13:24:56,307::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistant dict with FileMetadataRW backend
>Thread-51::DEBUG::2012-12-09 13:24:56,312::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=NFS_ISO_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=c34646d6-2dc7-11e2-8cb6-525400202852,4372535e-3f4d-11e2-8ba0-525400202852', 'REMOTE_PATH=mgr.rhev.icp.selfip.net:/var/lib/exports/iso', 'ROLE=Regular', 'SDUUID=c080b7a1-1d5f-449f-9a65-375961d37141', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=f0d093bcea4329aa26367556a4a04db827b67f62']
>Thread-51::DEBUG::2012-12-09 13:24:56,315::fileSD::447::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-51::WARNING::2012-12-09 13:24:56,315::sd::321::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c080b7a1-1d5f-449f-9a65-375961d37141_imageNS already registered
>Thread-51::WARNING::2012-12-09 13:24:56,315::sd::327::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c080b7a1-1d5f-449f-9a65-375961d37141_volumeNS already registered
>Thread-50::DEBUG::2012-12-09 13:24:58,246::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:24:58,261::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000400317 s, 10.2 MB/s\n'; <rc> = 0
>Thread-3586::DEBUG::2012-12-09 13:25:04,285::task::588::TaskManager.Task::(_updateState) Task=`73cb2ca7-e86a-44cc-b2c2-4aa44b6cd543`::moving from state init -> state preparing
>Thread-3586::INFO::2012-12-09 13:25:04,286::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3586::INFO::2012-12-09 13:25:04,286::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163669586182', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00222182273865', 'lastCheck': '8.0', 'code': 0, 'valid': True}}
>Thread-3586::DEBUG::2012-12-09 13:25:04,287::task::1172::TaskManager.Task::(prepare) Task=`73cb2ca7-e86a-44cc-b2c2-4aa44b6cd543`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0163669586182', 'lastCheck': '6.0', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00222182273865', 'lastCheck': '8.0', 'code': 0, 'valid': True}}
>Thread-3586::DEBUG::2012-12-09 13:25:04,287::task::588::TaskManager.Task::(_updateState) Task=`73cb2ca7-e86a-44cc-b2c2-4aa44b6cd543`::moving from state preparing -> state finished
>Thread-3586::DEBUG::2012-12-09 13:25:04,288::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3586::DEBUG::2012-12-09 13:25:04,288::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3586::DEBUG::2012-12-09 13:25:04,288::task::978::TaskManager.Task::(_decref) Task=`73cb2ca7-e86a-44cc-b2c2-4aa44b6cd543`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:08,265::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:08,278::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000916153 s, 4.5 MB/s\n'; <rc> = 0
>Thread-3592::DEBUG::2012-12-09 13:25:14,665::task::588::TaskManager.Task::(_updateState) Task=`c5c36ff1-a659-429a-b740-0fc0ca7df015`::moving from state init -> state preparing
>Thread-3592::INFO::2012-12-09 13:25:14,666::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3592::INFO::2012-12-09 13:25:14,667::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143961906433', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00235414505005', 'lastCheck': '8.3', 'code': 0, 'valid': True}}
>Thread-3592::DEBUG::2012-12-09 13:25:14,667::task::1172::TaskManager.Task::(prepare) Task=`c5c36ff1-a659-429a-b740-0fc0ca7df015`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143961906433', 'lastCheck': '6.4', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00235414505005', 'lastCheck': '8.3', 'code': 0, 'valid': True}}
>Thread-3592::DEBUG::2012-12-09 13:25:14,668::task::588::TaskManager.Task::(_updateState) Task=`c5c36ff1-a659-429a-b740-0fc0ca7df015`::moving from state preparing -> state finished
>Thread-3592::DEBUG::2012-12-09 13:25:14,668::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3592::DEBUG::2012-12-09 13:25:14,668::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3592::DEBUG::2012-12-09 13:25:14,669::task::978::TaskManager.Task::(_decref) Task=`c5c36ff1-a659-429a-b740-0fc0ca7df015`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:18,283::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:18,295::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000851872 s, 4.8 MB/s\n'; <rc> = 0
>Thread-3598::DEBUG::2012-12-09 13:25:25,072::task::588::TaskManager.Task::(_updateState) Task=`766d11ef-b5d7-4899-84ac-3b2745816ca4`::moving from state init -> state preparing
>Thread-3598::INFO::2012-12-09 13:25:25,072::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3598::INFO::2012-12-09 13:25:25,073::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120739936829', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00271511077881', 'lastCheck': '8.7', 'code': 0, 'valid': True}}
>Thread-3598::DEBUG::2012-12-09 13:25:25,073::task::1172::TaskManager.Task::(prepare) Task=`766d11ef-b5d7-4899-84ac-3b2745816ca4`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0120739936829', 'lastCheck': '6.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00271511077881', 'lastCheck': '8.7', 'code': 0, 'valid': True}}
>Thread-3598::DEBUG::2012-12-09 13:25:25,073::task::588::TaskManager.Task::(_updateState) Task=`766d11ef-b5d7-4899-84ac-3b2745816ca4`::moving from state preparing -> state finished
>Thread-3598::DEBUG::2012-12-09 13:25:25,073::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3598::DEBUG::2012-12-09 13:25:25,074::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3598::DEBUG::2012-12-09 13:25:25,074::task::978::TaskManager.Task::(_decref) Task=`766d11ef-b5d7-4899-84ac-3b2745816ca4`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:28,297::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:28,309::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00100648 s, 4.1 MB/s\n'; <rc> = 0
>Thread-3604::DEBUG::2012-12-09 13:25:35,610::task::588::TaskManager.Task::(_updateState) Task=`9f72f4d0-bed3-43af-92eb-1141e232992f`::moving from state init -> state preparing
>Thread-3604::INFO::2012-12-09 13:25:35,611::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3604::INFO::2012-12-09 13:25:35,611::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133512020111', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00242495536804', 'lastCheck': '9.3', 'code': 0, 'valid': True}}
>Thread-3604::DEBUG::2012-12-09 13:25:35,612::task::1172::TaskManager.Task::(prepare) Task=`9f72f4d0-bed3-43af-92eb-1141e232992f`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0133512020111', 'lastCheck': '7.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00242495536804', 'lastCheck': '9.3', 'code': 0, 'valid': True}}
>Thread-3604::DEBUG::2012-12-09 13:25:35,612::task::588::TaskManager.Task::(_updateState) Task=`9f72f4d0-bed3-43af-92eb-1141e232992f`::moving from state preparing -> state finished
>Thread-3604::DEBUG::2012-12-09 13:25:35,613::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3604::DEBUG::2012-12-09 13:25:35,613::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3604::DEBUG::2012-12-09 13:25:35,613::task::978::TaskManager.Task::(_decref) Task=`9f72f4d0-bed3-43af-92eb-1141e232992f`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:38,314::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:38,327::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00089495 s, 4.6 MB/s\n'; <rc> = 0
>Thread-3610::DEBUG::2012-12-09 13:25:45,827::task::588::TaskManager.Task::(_updateState) Task=`72045dbc-4dcb-40dc-b53d-bff0a78b717e`::moving from state init -> state preparing
>Thread-3610::INFO::2012-12-09 13:25:45,827::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3610::INFO::2012-12-09 13:25:45,828::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143609046936', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00271892547607', 'lastCheck': '9.5', 'code': 0, 'valid': True}}
>Thread-3610::DEBUG::2012-12-09 13:25:45,828::task::1172::TaskManager.Task::(prepare) Task=`72045dbc-4dcb-40dc-b53d-bff0a78b717e`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0143609046936', 'lastCheck': '7.5', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00271892547607', 'lastCheck': '9.5', 'code': 0, 'valid': True}}
>Thread-3610::DEBUG::2012-12-09 13:25:45,829::task::588::TaskManager.Task::(_updateState) Task=`72045dbc-4dcb-40dc-b53d-bff0a78b717e`::moving from state preparing -> state finished
>Thread-3610::DEBUG::2012-12-09 13:25:45,829::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3610::DEBUG::2012-12-09 13:25:45,830::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3610::DEBUG::2012-12-09 13:25:45,830::task::978::TaskManager.Task::(_decref) Task=`72045dbc-4dcb-40dc-b53d-bff0a78b717e`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:48,331::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:48,344::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000804814 s, 5.1 MB/s\n'; <rc> = 0
>Thread-3616::DEBUG::2012-12-09 13:25:56,117::task::588::TaskManager.Task::(_updateState) Task=`ba7d14f0-2905-41df-9580-8885f4dba4e0`::moving from state init -> state preparing
>Thread-3616::INFO::2012-12-09 13:25:56,118::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3616::INFO::2012-12-09 13:25:56,119::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134029388428', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00215005874634', 'lastCheck': '9.8', 'code': 0, 'valid': True}}
>Thread-3616::DEBUG::2012-12-09 13:25:56,119::task::1172::TaskManager.Task::(prepare) Task=`ba7d14f0-2905-41df-9580-8885f4dba4e0`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0134029388428', 'lastCheck': '7.8', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00215005874634', 'lastCheck': '9.8', 'code': 0, 'valid': True}}
>Thread-3616::DEBUG::2012-12-09 13:25:56,119::task::588::TaskManager.Task::(_updateState) Task=`ba7d14f0-2905-41df-9580-8885f4dba4e0`::moving from state preparing -> state finished
>Thread-3616::DEBUG::2012-12-09 13:25:56,119::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3616::DEBUG::2012-12-09 13:25:56,120::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3616::DEBUG::2012-12-09 13:25:56,120::task::978::TaskManager.Task::(_decref) Task=`ba7d14f0-2905-41df-9580-8885f4dba4e0`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:25:58,347::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:25:58,363::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000751071 s, 5.5 MB/s\n'; <rc> = 0
>Thread-3622::DEBUG::2012-12-09 13:26:06,485::task::588::TaskManager.Task::(_updateState) Task=`293c879f-7d04-4d59-8dc5-791be9b577c9`::moving from state init -> state preparing
>Thread-3622::INFO::2012-12-09 13:26:06,486::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3622::INFO::2012-12-09 13:26:06,487::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167770385742', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00208115577698', 'lastCheck': '0.1', 'code': 0, 'valid': True}}
>Thread-3622::DEBUG::2012-12-09 13:26:06,487::task::1172::TaskManager.Task::(prepare) Task=`293c879f-7d04-4d59-8dc5-791be9b577c9`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0167770385742', 'lastCheck': '8.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00208115577698', 'lastCheck': '0.1', 'code': 0, 'valid': True}}
>Thread-3622::DEBUG::2012-12-09 13:26:06,488::task::588::TaskManager.Task::(_updateState) Task=`293c879f-7d04-4d59-8dc5-791be9b577c9`::moving from state preparing -> state finished
>Thread-3622::DEBUG::2012-12-09 13:26:06,488::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3622::DEBUG::2012-12-09 13:26:06,488::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3622::DEBUG::2012-12-09 13:26:06,489::task::978::TaskManager.Task::(_decref) Task=`293c879f-7d04-4d59-8dc5-791be9b577c9`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:26:08,366::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:26:08,380::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00091236 s, 4.5 MB/s\n'; <rc> = 0
>Thread-3628::DEBUG::2012-12-09 13:26:16,728::task::588::TaskManager.Task::(_updateState) Task=`e2146d2f-7425-43f9-9ea3-f32a001b3863`::moving from state init -> state preparing
>Thread-3628::INFO::2012-12-09 13:26:16,729::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3628::INFO::2012-12-09 13:26:16,729::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0157680511475', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00251388549805', 'lastCheck': '0.3', 'code': 0, 'valid': True}}
>Thread-3628::DEBUG::2012-12-09 13:26:16,730::task::1172::TaskManager.Task::(prepare) Task=`e2146d2f-7425-43f9-9ea3-f32a001b3863`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0157680511475', 'lastCheck': '8.3', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00251388549805', 'lastCheck': '0.3', 'code': 0, 'valid': True}}
>Thread-3628::DEBUG::2012-12-09 13:26:16,730::task::588::TaskManager.Task::(_updateState) Task=`e2146d2f-7425-43f9-9ea3-f32a001b3863`::moving from state preparing -> state finished
>Thread-3628::DEBUG::2012-12-09 13:26:16,731::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3628::DEBUG::2012-12-09 13:26:16,731::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3628::DEBUG::2012-12-09 13:26:16,732::task::978::TaskManager.Task::(_decref) Task=`e2146d2f-7425-43f9-9ea3-f32a001b3863`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:26:18,384::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:26:18,400::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000936384 s, 4.4 MB/s\n'; <rc> = 0
>Thread-3634::DEBUG::2012-12-09 13:26:27,036::task::588::TaskManager.Task::(_updateState) Task=`910477d3-940a-4f5c-b0a0-885116c31427`::moving from state init -> state preparing
>Thread-3634::INFO::2012-12-09 13:26:27,036::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3634::INFO::2012-12-09 13:26:27,037::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016636133194', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00272703170776', 'lastCheck': '0.6', 'code': 0, 'valid': True}}
>Thread-3634::DEBUG::2012-12-09 13:26:27,037::task::1172::TaskManager.Task::(prepare) Task=`910477d3-940a-4f5c-b0a0-885116c31427`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.016636133194', 'lastCheck': '8.6', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00272703170776', 'lastCheck': '0.6', 'code': 0, 'valid': True}}
>Thread-3634::DEBUG::2012-12-09 13:26:27,037::task::588::TaskManager.Task::(_updateState) Task=`910477d3-940a-4f5c-b0a0-885116c31427`::moving from state preparing -> state finished
>Thread-3634::DEBUG::2012-12-09 13:26:27,037::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3634::DEBUG::2012-12-09 13:26:27,037::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3634::DEBUG::2012-12-09 13:26:27,038::task::978::TaskManager.Task::(_decref) Task=`910477d3-940a-4f5c-b0a0-885116c31427`::ref 0 aborting False
>Thread-50::DEBUG::2012-12-09 13:26:28,404::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:26:28,415::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000587175 s, 7.0 MB/s\n'; <rc> = 0
>Thread-3640::DEBUG::2012-12-09 13:26:37,548::task::588::TaskManager.Task::(_updateState) Task=`1ff934da-7e22-4663-a98f-df632ea78cc6`::moving from state init -> state preparing
>Thread-3640::INFO::2012-12-09 13:26:37,549::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-3640::INFO::2012-12-09 13:26:37,549::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114130973816', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00283908843994', 'lastCheck': '1.2', 'code': 0, 'valid': True}}
>Thread-3640::DEBUG::2012-12-09 13:26:37,549::task::1172::TaskManager.Task::(prepare) Task=`1ff934da-7e22-4663-a98f-df632ea78cc6`::finished: {u'c294c8b1-b2e7-4564-bbb2-2d6c1f54b895': {'delay': '0.0114130973816', 'lastCheck': '9.1', 'code': 0, 'valid': True}, u'c080b7a1-1d5f-449f-9a65-375961d37141': {'delay': '0.00283908843994', 'lastCheck': '1.2', 'code': 0, 'valid': True}}
>Thread-3640::DEBUG::2012-12-09 13:26:37,549::task::588::TaskManager.Task::(_updateState) Task=`1ff934da-7e22-4663-a98f-df632ea78cc6`::moving from state preparing -> state finished
>Thread-3640::DEBUG::2012-12-09 13:26:37,549::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3640::DEBUG::2012-12-09 13:26:37,550::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3640::DEBUG::2012-12-09 13:26:37,550::task::978::TaskManager.Task::(_decref) Task=`1ff934da-7e22-4663-a98f-df632ea78cc6`::ref 0 aborting False
>Thread-3642::DEBUG::2012-12-09 13:26:37,649::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3642::DEBUG::2012-12-09 13:26:37,649::task::588::TaskManager.Task::(_updateState) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::moving from state init -> state preparing
>Thread-3642::INFO::2012-12-09 13:26:37,649::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='4372535e-3f4d-11e2-8ba0-525400202852', hostID=2, scsiKey='4372535e-3f4d-11e2-8ba0-525400202852', remove=False, options=None)
>Thread-3642::DEBUG::2012-12-09 13:26:37,650::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a5974d5c-81d5-425c-94a3-f9ec2a2662cd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource'
>Thread-3642::DEBUG::2012-12-09 13:26:37,650::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' for lock type 'exclusive'
>Thread-3642::DEBUG::2012-12-09 13:26:37,650::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free. Now locking as 'exclusive' (1 active user)
>Thread-3642::DEBUG::2012-12-09 13:26:37,650::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4372535e-3f4d-11e2-8ba0-525400202852`ReqID=`a5974d5c-81d5-425c-94a3-f9ec2a2662cd`::Granted request
>Thread-3642::DEBUG::2012-12-09 13:26:37,651::task::817::TaskManager.Task::(resourceAcquired) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::_resourcesAcquired: Storage.4372535e-3f4d-11e2-8ba0-525400202852 (exclusive)
>Thread-3642::DEBUG::2012-12-09 13:26:37,651::task::978::TaskManager.Task::(_decref) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::ref 1 aborting False
>Thread-3642::INFO::2012-12-09 13:26:37,651::sp::668::Storage.StoragePool::(disconnect) Disconnect from the storage pool 4372535e-3f4d-11e2-8ba0-525400202852
>Thread-58::INFO::2012-12-09 13:26:37,796::storage_mailbox::446::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail
>Thread-58::INFO::2012-12-09 13:26:37,796::storage_mailbox::340::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16']
>Thread-58::DEBUG::2012-12-09 13:26:37,797::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/rhev/data-center/4372535e-3f4d-11e2-8ba0-525400202852/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16' (cwd None)
>Thread-58::DEBUG::2012-12-09 13:26:37,814::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00437741 s, 936 kB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:26:38,418::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd iflag=direct if=/dev/c294c8b1-b2e7-4564-bbb2-2d6c1f54b895/metadata bs=4096 count=1' (cwd None)
>Thread-50::DEBUG::2012-12-09 13:26:38,433::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000754841 s, 5.4 MB/s\n'; <rc> = 0
>Thread-50::DEBUG::2012-12-09 13:26:39,591::domainMonitor::153::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for c294c8b1-b2e7-4564-bbb2-2d6c1f54b895
>Thread-50::INFO::2012-12-09 13:26:39,592::safelease::183::SANLock::(releaseHostId) Releasing host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 (id: 2)
>Thread-50::DEBUG::2012-12-09 13:26:41,593::safelease::193::SANLock::(releaseHostId) Host id for domain c294c8b1-b2e7-4564-bbb2-2d6c1f54b895 released successfully (id: 2)
>Thread-51::DEBUG::2012-12-09 13:26:41,594::domainMonitor::153::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for c080b7a1-1d5f-449f-9a65-375961d37141
>Thread-3642::INFO::2012-12-09 13:26:41,595::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True
>Thread-3642::DEBUG::2012-12-09 13:26:41,595::task::1172::TaskManager.Task::(prepare) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::finished: True
>Thread-3642::DEBUG::2012-12-09 13:26:41,595::task::588::TaskManager.Task::(_updateState) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::moving from state preparing -> state finished
>Thread-3642::DEBUG::2012-12-09 13:26:41,596::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4372535e-3f4d-11e2-8ba0-525400202852': < ResourceRef 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', isValid: 'True' obj: 'None'>}
>Thread-3642::DEBUG::2012-12-09 13:26:41,596::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3642::DEBUG::2012-12-09 13:26:41,597::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852'
>Thread-3642::DEBUG::2012-12-09 13:26:41,597::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' (0 active users)
>Thread-3642::DEBUG::2012-12-09 13:26:41,598::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852' is free, finding out if anyone is waiting for it.
>Thread-3642::DEBUG::2012-12-09 13:26:41,598::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4372535e-3f4d-11e2-8ba0-525400202852', Clearing records.
>Thread-3642::DEBUG::2012-12-09 13:26:41,599::task::978::TaskManager.Task::(_decref) Task=`2772ea88-2849-4a31-8a8f-6cc60f02a580`::ref 0 aborting False
>Thread-3643::DEBUG::2012-12-09 13:26:41,666::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3643::DEBUG::2012-12-09 13:26:41,667::task::588::TaskManager.Task::(_updateState) Task=`0bb3a47d-c341-4e06-a4e7-6152e70bb42c`::moving from state init -> state preparing
>Thread-3643::INFO::2012-12-09 13:26:41,667::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID='4372535e-3f4d-11e2-8ba0-525400202852', conList=[{'connection': '192.168.0.254', 'iqn': 'iqn.2012-11.net.selfip.icp:ian.1', 'portal': '1', 'user': '', 'password': '******', 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3', 'port': '3260'}, {'connection': '192.168.1.254', 'iqn': 'iqn.2012-11.net.selfip.icp:ian.1', 'portal': '1', 'user': '', 'password': '******', 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf', 'port': '3260'}], options=None)
>MainProcess|Thread-3643::ERROR::2012-12-09 13:26:41,668::supervdsmServer::68::SuperVdsm.ServerCallback::(wrapper) Error in readSessionInfo
>Traceback (most recent call last):
>  File "/usr/share/vdsm/supervdsmServer.py", line 66, in wrapper
>    return func(*args, **kwargs)
>  File "/usr/share/vdsm/supervdsmServer.py", line 98, in readSessionInfo
>    return _readSessionInfo(sessionID)
>  File "/usr/share/vdsm/storage/iscsi.py", line 83, in readSessionInfo
>    raise OSError(errno.ENOENT, "No such session")
>OSError: [Errno 2] No such session
>Thread-3643::DEBUG::2012-12-09 13:26:41,720::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -r 2 -u' (cwd None)
>Thread-3643::DEBUG::2012-12-09 13:26:42,573::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>MainProcess|Thread-3643::ERROR::2012-12-09 13:26:42,574::supervdsmServer::68::SuperVdsm.ServerCallback::(wrapper) Error in readSessionInfo
>Traceback (most recent call last):
>  File "/usr/share/vdsm/supervdsmServer.py", line 66, in wrapper
>    return func(*args, **kwargs)
>  File "/usr/share/vdsm/supervdsmServer.py", line 98, in readSessionInfo
>    return _readSessionInfo(sessionID)
>  File "/usr/share/vdsm/storage/iscsi.py", line 83, in readSessionInfo
>    raise OSError(errno.ENOENT, "No such session")
>OSError: [Errno 2] No such session
>Thread-3643::DEBUG::2012-12-09 13:26:42,576::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -r 1 -u' (cwd None)
>Thread-3643::DEBUG::2012-12-09 13:26:43,423::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3643::DEBUG::2012-12-09 13:26:43,423::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-3643::DEBUG::2012-12-09 13:26:43,424::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3643::DEBUG::2012-12-09 13:26:43,424::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-3643::DEBUG::2012-12-09 13:26:43,424::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3643::DEBUG::2012-12-09 13:26:43,424::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-3643::DEBUG::2012-12-09 13:26:43,438::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-3643::DEBUG::2012-12-09 13:26:43,438::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-3643::DEBUG::2012-12-09 13:26:43,439::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-3643::DEBUG::2012-12-09 13:26:43,442::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-3643::DEBUG::2012-12-09 13:26:43,446::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-3643::DEBUG::2012-12-09 13:26:45,449::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-3643::DEBUG::2012-12-09 13:26:45,480::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3643::DEBUG::2012-12-09 13:26:45,481::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,482::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,482::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,483::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,483::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,483::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3643::DEBUG::2012-12-09 13:26:45,484::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-3643::INFO::2012-12-09 13:26:45,484::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3'}, {'status': 0, 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf'}]}
>Thread-3643::DEBUG::2012-12-09 13:26:45,484::task::1172::TaskManager.Task::(prepare) Task=`0bb3a47d-c341-4e06-a4e7-6152e70bb42c`::finished: {'statuslist': [{'status': 0, 'id': '7da9b5e4-cef4-42ae-891f-2de26fc91bb3'}, {'status': 0, 'id': 'abca0658-70db-4925-86bc-5ecedcda1fcf'}]}
>Thread-3643::DEBUG::2012-12-09 13:26:45,484::task::588::TaskManager.Task::(_updateState) Task=`0bb3a47d-c341-4e06-a4e7-6152e70bb42c`::moving from state preparing -> state finished
>Thread-3643::DEBUG::2012-12-09 13:26:45,484::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3643::DEBUG::2012-12-09 13:26:45,485::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3643::DEBUG::2012-12-09 13:26:45,485::task::978::TaskManager.Task::(_decref) Task=`0bb3a47d-c341-4e06-a4e7-6152e70bb42c`::ref 0 aborting False
>Thread-3644::DEBUG::2012-12-09 13:26:45,520::BindingXMLRPC::171::vds::(wrapper) [172.31.249.200]
>Thread-3644::DEBUG::2012-12-09 13:26:45,521::task::588::TaskManager.Task::(_updateState) Task=`90081716-4220-4c2f-a977-c233c76ea345`::moving from state init -> state preparing
>Thread-3644::INFO::2012-12-09 13:26:45,521::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='4372535e-3f4d-11e2-8ba0-525400202852', conList=[{'port': '', 'connection': 'mgr.rhev.icp.selfip.net:/var/lib/exports/iso', 'iqn': '', 'portal': '', 'user': '', 'protocol_version': '4', 'password': '******', 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}], options=None)
>Thread-3644::DEBUG::2012-12-09 13:26:45,521::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/mgr.rhev.icp.selfip.net:_var_lib_exports_iso' (cwd None)
>Thread-3644::DEBUG::2012-12-09 13:26:45,544::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)
>Thread-3644::DEBUG::2012-12-09 13:26:45,550::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3644::DEBUG::2012-12-09 13:26:45,550::misc::1044::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)
>Thread-3644::DEBUG::2012-12-09 13:26:45,551::misc::1046::SamplingMethod::(__call__) Got in to sampling method
>Thread-3644::DEBUG::2012-12-09 13:26:45,551::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
>Thread-3644::DEBUG::2012-12-09 13:26:45,567::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21
>Thread-3644::DEBUG::2012-12-09 13:26:45,567::misc::1054::SamplingMethod::(__call__) Returning last result
>MainProcess|Thread-3644::DEBUG::2012-12-09 13:26:45,570::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host0/scan' (cwd None)
>MainProcess|Thread-3644::DEBUG::2012-12-09 13:26:46,575::__init__::1164::Storage.Misc.excCmd::(_log) '/bin/dd of=/sys/class/scsi_host/host1/scan' (cwd None)
>MainProcess|Thread-3644::DEBUG::2012-12-09 13:26:47,581::iscsi::388::Storage.ISCSI::(forceIScsiScan) Performing SCSI scan, this will take up to 30 seconds
>Thread-3644::DEBUG::2012-12-09 13:26:49,584::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None)
>Thread-3644::DEBUG::2012-12-09 13:26:49,617::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
>Thread-3644::DEBUG::2012-12-09 13:26:49,617::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,617::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,618::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,618::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,618::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,618::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-3644::DEBUG::2012-12-09 13:26:49,618::misc::1054::SamplingMethod::(__call__) Returning last result
>Thread-3644::INFO::2012-12-09 13:26:49,619::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3644::DEBUG::2012-12-09 13:26:49,619::task::1172::TaskManager.Task::(prepare) Task=`90081716-4220-4c2f-a977-c233c76ea345`::finished: {'statuslist': [{'status': 0, 'id': '0d125a37-cae4-4f1e-a6f6-a41b8fde8c43'}]}
>Thread-3644::DEBUG::2012-12-09 13:26:49,619::task::588::TaskManager.Task::(_updateState) Task=`90081716-4220-4c2f-a977-c233c76ea345`::moving from state preparing -> state finished
>Thread-3644::DEBUG::2012-12-09 13:26:49,619::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-3644::DEBUG::2012-12-09 13:26:49,619::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-3644::DEBUG::2012-12-09 13:26:49,619::task::978::TaskManager.Task::(_decref) Task=`90081716-4220-4c2f-a977-c233c76ea345`::ref 0 aborting False

Actions: View

Attachments on bug 885489: 660406 | 660407 | 660408